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Abstract 


The  primary  intent  of  this  study  was  to  explore  the  effect  of  a Computer  Assisted 
Instruction  (CAI)  system  on  student  learning  and  achievement.  The  CAI  system 
under  study  is  an  integrated  learning  system  known  as  DEGEM.  The  system 
consists  of  a 40  megabyte  miniframe  computer  and  24  terminals  with  detachable 
keyboards.  This  system  was  studied  in  two  Canadian  Native  schools  in 
Northeastern  Alberta.  Students  from  grades  two  through  eight  participated  in  the 
longitudinal  study,  which  involved  the  collection  of  both  quantitative  and  qualitative 
data  from  both  sites. 

Eight  research  questions  were  posed,  two  requiring  quantitative  data,  involved 
exploring  student  achievement  and  thinking  skills.  Questions  regarding  the 
diagnostic  capabilities  of  the  system  and  its  application  to  special  needs  students 
were  explored  using  a case  study  format.  The  area  of  the  suitability  and  reliability 
of  the  hardware  and  software  was  also  explored.  Qualitative  data  was  used  to 
examine  the  need  for  teacher  inservice  necessary  to  use  the  systems,  as  well  as  to 
explore  student,  parent,  teacher,  and  administrator  perceptions  of  the  system. 
Finally,  student  time  on  task  was  examined  while  on  the  system  and  in  the 
classroom.  Each  question  was  examined  in  detail,  data  were  presented,  and  a brief 
discussion  ensued. 

There  were  a number  of  problems  in  applying  the  original  study  design  which  was 
supplied  to  the  researchers.  Attempts  were  made  to  minimize  any  threats  to 
validity.  Unfortunately,  due  to  a number  of  difficulties  experienced  at  both  sites 
very  little  quantitative  data  was  available  from  which  to  develop  conclusions  or 
make  recommendations.  For  the  most  part  the  qualitative  data  was  intact  and 
yielded  some  insights  into  the  DEGEM  system.  These  areas  of  difficulty  are 
discussed  in  some  detail  in  an  attempt  to  assist  and  guide  future  researchers. 

The  study  concludes  with  several  recommendations  including  the  need  for  a very 
knowledgeable  systems  manager,  the  need  for  ongoing  and  intensive  inservice,  and 
the  need  for  further  study  of  the  system's  diagnostic  capabilities.  The  Curriculum 
Support  Branch's  appraisal  of  the  DEGEM  system  concludes  with  a 
recommendation  that  the  system  is  not  suitable  for  general  use  in  Alberta. 


in  2016 
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CHAPTER  1 - INTRODUCTION 


The  DEGEM  system  is  an  integrated  learning  system  distributed  by  Innovative 
Technologies  in  Education  Inc.  In  both  Onchaminahos  and  Kehewin  schools,  the 
system  consists  of  a 40  megabyte  miniframe  computer  with  a 20  megabyte  tape 
back  up,  24  terminals  with  detachable  keyboards,  one  printer,  TOAM  2A 
courseware  and  additional  software  for  systems  management  and  authoring.  At 
Onchaminahos  School  the  system  is  referred  to  as  "DEGEM",  while  at  Kehewin 
School,  the  system  is  referred  to  as  "TOAM".  To  avoid  confusion  throughout  this 
report,  the  system  will  be  referred  to  as  DEGEM. 

The  TOAM  2A  courseware,  developed  in  Israel,  provides  basic  skill  and  drill 
exercises  in  mathematics  for  grades  one  through  eight  inclusive,  and  in  language 
arts  for  grades  two  through  seven  inclusive  (DEGEM,  1987).  In  addition,  the 
system  has  extensive  recordkeeping  functions  which  enable  the  user  to  obtain  a 
number  of  diagnostic  printouts,  by  class,  for  either  language  arts  or  mathematics. 
The  printout,  through  a variety  of  symbolic  codes,  enables  the  teacher  to  track  any 
given  student's  progress  throughout  the  course. 

'The  system  at  Kehewin  School  was  installed  in  the  fall  of  1986.  Students  in  grades 
two  through  nine  were  involved  in  regularly  using  the  system,  until  June  1988 
primarily  for  mathematics,  although  some  classes  used  it  for  language  arts  as  well. 
A few  classes  were  involved  in  other  activities  such  as  typing.  In  the  1988-1989 
school  year  there  were  a number  of  changes  at  the  school  which  impacted  the  use  of 
the  system.  There  were  two  administrative  changes  (September  1988;  January 
1989);  two  systems  manager  changes  (September  1988;  February  1989);  and 
almost  half  the  teachers  who  were  there  in  June  1988  were  not  there  in  September 
1988.  As  a result,  in  1988-89  it  appears  that  the  DEGEM  system  was  used 
sporadically  until  the  January  - February  time  frame. 
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At  Onchaminahos  School,  the  system  was  installed  in  January  1988.  Students 
from  that  time  until  June  1989  used  the  system  regularly.  Grades  K through  nine 
used  the  system  for  mathematics,  language  arts,  and  other  activities.  The  systems 
manager  at  Onchaminahos  resigned  in  June  1989.  At  that  time  it  was  requested  that 
he  shut  the  system  down  and  lock  the  drives,  as  one  would  do  to  prepare  to  ship  the 
computer  system.  Apparently,  at  this  time  there  are  no  plans  to  use  the  system  for 
the  1989-1990  school  year. 

The  computer  systems  were  purchased  to  provide  assistance  to  the  students  in  both 
schools  to  enhance  their  academic  skills  in  language  arts  and  mathematics. 
Gittinger  (1986)  indicates  that  the  DEGEM  courseware  was  developed  to  provide, 
"a  well  sequenced  instructional  support  system  in  language  and  mathematics  . . . 
[for]  culturally  diverse  immigrants.  ...  the  program  was  intended  to  have  its 
greatest  impact  in  rural  areas  suffering  from  teacher  turnover  and  relative 
deprivation  due  to  isolation."  (p.  2).  Students  involved  in  the  present  study  are  not 
immigrants;  they  are,  however,  part  of  a Native  culture  and  are  in  schools  which  are 
relatively  isolated.  Rumage  (1987)  indicates  that  DEGEM  has  been  used 
successfully  in  a small  isolated  school  district  in  Texas.  Additional  literature  has 
been  reviewed,  some  of  which  appears  to  have  been  written  "in  house",  (i.e.,  by 
DEGEM  staff,  evaluating  the  system);  other  literature  has  been  published  in  several 
different  journals.  It  does  not  appear,  however,  that  a comprehensive  literature 
review  has  been  conducted  with  respect  to  the  DEGEM  system,  and  although  it  falls 
outside  the  scope  of  this  report,  the  results  of  such  a review  provide  additional 
insight  into  the  effectiveness  of  the  DEGEM  system. 

This  final  report  is  divided  into  three  major  sections:  introduction;  results  and 
discussion;  and  summary  and  recommendations.  The  results  and  discussion 
section  is  further  divided  into  eight  components,  identified  by  each  research 
question.  Eight  questions  (numbers  1,  3-9)  and  the  basic  research  design  were 
given  to  MacRosch  Educational  Consulting  Services  Inc.  by  Alberta  Education. 
One  question,  question  two,  was  to  be  answered  by  Alberta  Education.  The 
findings  can  be  found  in  Appendix  C.  There  were  some  initial  concerns  raised  by 
MacRosch  as  to  whether  the  design  would  allow  for  the  collection  of  data  necessary 
to  accurately  answer  some  of  the  questions  posed.  For  example,  the  heart  of  the 
study  required  the  measurement  of  gains  that  students  made  in  either  language  arts 
or  mathematics  when  using  the  DEGEM  system.  A control  group  was  part  of  the 
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initial  design;  however,  the  "control  group"  specified  by  the  design  also  used  the 
DEGEM  system,  just  in  another  subject  area.  This  design  was  based  on  the 
assumption  that  using  the  system  would  not  generalize  across  subject  areas; 
however,  there  would  be  no  way  of  ascertaining  wether  or  not  this  would  be  the 
case  as  there  was  no  "control  group"  who  did  not  use  the  DEGEM  system  at  all. 
Another  weakness  in  this  area  was  that  because  of  the  size  of  the  student  enrolment, 
the  design  required  that  the  control  group  for  the  grade  four  group,  for  example, 
was  the  grade  five  group.  These  types  of  threats  to  the  internal  validity  to  the  study 
were  inherent  in  the  original  design.  It  was  decided  by  the  Steering  Committee,  that 
given  the  available  sample,  the  project  should  proceed  as  tendered. 

The  primary  intent  of  this  study  is  to  determine  whether  using  the  DEGEM  system 
in  combination  with  regular  classroom  instruction  enhances  student  learning  in 
mathematics  and  language  arts  over  regular  classroom  instruction  alone.  It  appears 
that  most  of  the  studies  conducted  to  date  regarding  student  achievement  have  been 
coordinated  by  DEGEM  staff  and  have  been  conducted  over  periods  ranging  from 
two  weeks  to  several  months.  At  this  time  it  does  not  appear  that  any  studies  one  or 
more  years  in  length  have  been  conducted. 

This  next  section  describes  both  the  qualitative  and  quantitative  data  collected. 
Results  are  reported  for  each  question  given  in  the  original  design,  beginning  with 
question  one.  The  results  of  Alberta  Education's  findings  for  question  two,  dealing 
with  the  appropriateness  of  the  courseware  relative  to  Alberta  Education 
Clearinghouse  criteria  and  standards,  are  reported  as  given  to  MacRosch  in 
Appendix  C.  The  questions  with  respect  to  whether  DEGEM  CAI  instruction 
results  in  improvements  in  student  achievement  (question  one),  and  thinking  skills 
(question  five),  combine  qualitative  and  quantitative  data.  The  DEGEM  diagnostics 
compared  to  the  CAT  diagnostics  (question  three),  and  use  in  a special  education 
context  (question  nine),  are  presented  in  a case  study  format.  This  format  yields  the 
most  information  given  the  design  and  contextual  limitations  which  will  be 
discussed  later.  The  data  reported  in  the  area  of  inservice  training  (question  four), 
are  presented  for  the  1987-1988  school  year  only,  as  no  inservice  was  conducted 
during  the  subsequent  academic  year.  The  teacher  and  administrator  perceptions 
(question  six),  and  student  perceptions  (question  seven),  are  exclusively  qualitative 
and  derived  from  responses  to  questionnaires  and  structured  interviews.  The  cost 
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effectiveness  and  system  reliability  (question  eight)  data  were  gathered  from  several 
sources  and  will  be  reported  as  collected. 

Questionnaires  were  distributed  to  teachers,  students,  administrators,  and  systems 
managers.  The  results  arising  from  the  questionnaires  are  reported  in  the  same 
manner  across  groups.  Respondents  were  asked  questions  and  selected  their 
responses  from  a five-point  rating  scale:  0 = strongly  disagree;  1 = disagree; 
2 = neutral;  3 = agree;  4 = strongly  agree. 

The  original  design  called  for  a qualitative  assessment  of  DEGEM  at  both  sites. 
The  information  was  to  be  gathered  from  teachers,  students,  systems  managers, 
parents,  and  administration.  This  was  done  in  1988.  Questionnaires  were 
distributed  to  students  in  both  schools,  Onchaminahos  grades  2 through  6 inclusive; 
Kehewin  grades  2 through  7 inclusive.  All  teachers  who  taught  the  same  grades  as 
the  students  surveyed,  were  given  questionnaires.  The  systems  manager  from  each 
school  was  given  a questionnaire  as  were  the  administrators  (3  at  Onchaminahos 
and  1 at  Kehewin).  Face  to  face  interviews  were  conducted  with  a select  sample  of 
parents,  a random  sample  of  students,  and  all  teachers,  systems  managers,  and 
administrators.  In  1989  the  Steering  Committee  decided  that,  because  of 
community  sensitivity,  the  parent  group  would  not  be  interviewed  again,  and  that 
the  questionnaires  given  to  the  systems  managers,  teachers,  and  administrators 
would  be  replaced  by  the  teacher  questionnaire,  which  would  be  given  to  all 
groups.  The  teacher  questionnaire  was  modified,  in  that  all  references  to  inservice 
were  deleted,  as  no  inservice  had  been  conducted  between  May  1988  and  June 
1989.  All  teachers  again  were  invited  to  participate  in  a face  to  face  interview.  No 
face  to  face  interviews  were  to  be  conducted  with  students  unless  some  unusual 
findings  were  to  arise  out  of  the  second  round  of  questionnaires. 

With  respect  to  participation  in  the  qualitative  portion  of  this  study,  in  1988,  eight 
teachers  from  Kehewin  School  and  nine  teachers  form  Onchaminahos  School  wrote 
comments  on  the  questionnaires.  In  the  same  year,  89  students  from  Kehewin  and 
109  students  from  Onchaminahos  responded  to  the  comments  section  of  their 
questionnaire.  Face  to  face  interviews  were  conducted  with  eight  teachers  from 
Kehewin  and  nine  teachers  from  Onchaminahos.  In  addition,  interviews  were 
conducted  with  33  students  from  both  Kehewin  and  Onchaminahos.  A total  of 
seven  parents  were  interviewed  from  both  schools  combined.  In  1989,  nine 
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teachers  from  Onchaminahos  returned  the  answer  sheet  to  the  questionnaire; 
however,  no  written  comments  were  submitted.  At  Kehewin  two  teachers  returned 
the  answer  sheet;  however,  one  was  incomplete.resulting  in  insufficient  data  to  be 
considered  representative  of  the  perceptions  of  the  Kehewin  staff.  It  should  also  be 
noted  that  given  the  very  high  staff  turnover  in  the  school,  the  results  from  the  first 
round  of  data  collection  to  the  second  would  not  have  been  comparable,  even  if  all 
the  Kehewin  teachers  had  completed  the  questionnaire.  In  1989,  87  students  at 
Kehewin  completed  questionnaires  while  ninety-five  students  at  Onchaminahos  also 
completed  questionnaires.  In  1989,  the  systems  managers  and  administrators 
competed  the  teacher  questionnaire  rather  than  their  own  specific  questionnaire  as 
they  did  in  1988.  Also  in  1989  no  parent  interviews  were  conducted  nor  were  any 
interviews  with  students  conducted;  however,  8 teachers  were  interviewed  at 
Onchaminahos  and  9 teachers  were  interviewed  at  Kehewin. 

One  serious  problem  arose  during  the  course  of  collecting  the  quantitative  data.  The 
data  collected  at  Onchaminahos  were  contaminated  during  the  second 
round,. primarily  due  to  the  fact  that  the  student  cohort  groups  were  mixed.  An 
example  of  how  serious  the  contamination  was  will  be  given  for  the  first  two  grades 
which  comprised  the  study.  In  1988  the  original  design  called  for  one  class  of 
grade  2's  (2A),  to  use  the  DEGEM  system  exclusively  for  mathematics  and  the 
grade  2B  class  to  use  the  system  exclusively  for  English.  As  this  group  moved  to 
grade  three,  the  3A  class  was  to  remain  on  mathematics  and  the  3B  class  on 
English.  In  1988, 18  students  were  in  2A  and  18  students  were  in  2B,  for  a total  of 
36  students.  In  1989,  there  were  12  students  in  each  of  2A  and  2B,  for  a total  of 
24  students.  When  the  common  students  are  abstracted  from  the  two  groups  there 
were  a total  of  16  students  who  were  common  (9  in  mathematics  and  7 in  English). 
All  other  students  were  in  these  classes  for  only  one  year  or  the  other.  When  the 
students  are  tracked  more  carefully  it  will  be  found  that  of  the  eighteen  students 
who  started  in  mathematics,  by  grade  three,  seven  of  them  had  been  moved  to  the 
English  class  and  were  subsequently  lost  to  the  study.  Of  the  18  students  who 
started  in  English,  five  had  been  moved  to  mathematics.  The  net  result  is,  of  the  36 
students  who  began  the  study  in  grade  2,  four  useable  subjects  remained  in  English 
and  four  useable  subjects  remained  in  mathematics.  These  numbers  could  be 
further  reduced  if  any  of  these  eight  students  did  not  complete  the  total  CAT  or  TCS 
battery.  With  respect  to  the  movement  of  students  from  one  group  to  the  other,  it 
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was  understood  by  the  school  administration  at  the  outset  that  student  needs  would 
take  precedence  over  the  study  design. 

In  1988  the  grade  three  class,  there  were  originally  38  students  divided  between 
mathematics  and  English.  In  1989  these  two  classes  were  combined  into  one  grade 
four  class.  This  combined  class  used  the  system  exclusively  for  English;  therefore, 
all  the  mathematics  subjects  were  lost.  Late  in  the  1988-1989  school  year  this  class 
was  split  into  two  classes.  Unfortunately,  when  the  class  was  split,  the  students 
were  not  divided  such  that  they  used  the  system  for  the  same  subjects  as  they  did  in 
1988  (i.e.,  some  of  the  English  students  were  placed  in  the  mathematics  section). 
When  the  common  students  over  two  years  are  abstracted  and  the  numbers  are 
adjusted  for  who  switched  subjects  on  the  system,  there  are  no  useable  subjects 
remaining  in  the  current  grade  four  class. 

These  two  groups  were  the  backbone  of  the  study.  When  they  were  lost  it  limited 
the  utility  of  the  quantitative  data  in  part  two.  As  well,  problems  with  decreasing 
numbers  and  incomplete  data  were  evident  in  the  remaining  grade  levels,  rendering 
the  data  inconclusive,  at  best. 


CHAPTER  2 - RESULTS  AND  DISCUSSION 


Question  One  - Does  the  use  of  DEGEM  CAI  result  in  improvements  in  student 
achievement  in  reading  and  mathematics  relative  to  conventional  instruction?  What 
measured  gains  in  reading  and  math  occurred? 

As  was  indicated  in  the  original  design,  the  quantitative  results  required  to  answer 
this  question  were  to  be  gathered  from  Onchaminahos  School  in  Saddle  Lake.  The 
instrument  used  to  gather  the  data  was  the  Canadian  Achievement  Test  (CAT).  As 
this  test  was  not  being  used  in  either  school  (Onchaminahos  or  Kehewin)  prior  to 
the  study,  an  inservice  was  conducted  with  each  staff.  Generally  the  inservice 
covered  topics  such  as  how  to  administer  a standardized  test,  the  strengths  and 
weaknesses  of  the  CAT,  and  the  purpose  of  the  testing.  The  administration  of  the 
CAT  was  conducted  by  the  classroom  teacher  during  a ten-day  interval  in  May 
1988,  and  subsequently  in  early  June  1989.  The  results  were  collected  and  scored. 
The  numbers  reported  at  each  grade  level  indicate  the  total  number  of  students  who 
wrote  the  CAT.  Unfortunately,  in  many  cases  there  were  missing  data  (i.e.,  the 
child  only  wrote  parts  of  the  CAT);  therefore,  the  actual  records  read  for  any  one 
subtest  could  vary  considerably. 

The  1988  results  comparing  Grade  2A  with  Grade  2B,  Grade  3A  with  Grade  3B, 
Grade  4 with  Grade  5,  and  Grade  5 with  Grade  6 are  reported  in  Table  1.  The 
Grade  7 class  was  not  included  in  this  study  as  they  were  not  using  the  DEGEM 
system  in  1988.  The  results  reported  are  based  on  the  average  raw  scores  for  each 
class  on  five  measures;  total  reading,  spelling,  total  language,  total  mathematics, 
and  total  battery. 
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Table  1 

Mean  Achievement  SQQrs$  pn  thg  Achievement  Rgppne^^  By 

Grade  2A  (Mathematics)  By  Grade  2B  (Language) 


Area 

Means 

Standard  Dev. 

T 

Prob.2-Tail 

2A 

2B 

2A 

2B 

Reading 

58.1 

39.5 

9.9 

12.6 

4.7 

0.00 

Spelling 

15.5 

12.2 

3.5 

2.7 

3.1 

0.00 

Language 

31.3 

22.4 

5.0 

5.4 

4.8 

0.00 

Mathematics 

52.8 

39.3 

8.6 

11.1 

3.8 

0.00 

TOTAL 

161.6 

113.4 

16.2 

28.0 

5.7 

0.00 

Grade  3A  (Mathematics)  By  Grade  3B  (Language) 


Area 

Means 

Standard  Dev. 

T 

Prob.  2-Tail 

3A 

3B 

3A 

3B 

Reading 

32.9 

44.0 

11.4 

12.0 

-2.6 

0.01 

Spelling 

12.6 

12.2 

3.5 

3.8 

0.0 

0.97 

Language 

23.1 

23.9 

5.1 

6.3 

-0.4 

0.68 

Mathematics 

41.4 

45.5 

13.8 

13.6 

1 

o 

\D 

0.40 

TOTAL 

107.9 

125.2 

27.3 

30.2 

-1.6 

0.13 

Grade  4 (Mathematics)  By  Grade  5 (Language) 


Area 

Means 

Standard  Dev. 

T 

Prob.  2-Tail 

4 

5 

4 

5 

Reading 

28.6 

30.3 

9.4 

11.3 

-0.5 

0.62 

Spelling 

12.2 

12.9 

4.2 

3.9 

-0.6 

0.56 

Language 

29.5 

29.6 

8.6 

10.6 

1 

p 

b 

0.99 

Mathematics 

35.8 

34.5 

10.5 

12.7 

0.4 

0.73 

TOTAL 

107.2 

106.1 

26.5 

29.5 

0.1 

0.90 
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Table  1 Continued 

M.gan  Aghiey£ment..Scprg§  QP  the  Canadian  Aghigvgmgnt  Tg$t  Repined  By  Qrade 
Grade  5 (Mathematics)  By  Grade  6 (Language) 


Area 

Means 

Standard  Dev. 

T 

Prob. 

5 

6 

5 

6 

Reading 

30.3 

31.9 

11.3 

10.0 

-0.5 

0.64 

Spelling 

12.9 

14.8 

3.9 

3.0 

-1.6 

0.12 

Language 

29.6 

33.8 

10.6 

9.1 

-1.3 

0.21 

Mathematics 

34.5 

55.6 

12.7 

14.0 

-4.8 

0.00 

TOTAL 

106.1 

136.5 

29.5 

31.9 

-2.9 

0.01 

Grade  2 A (n=18)  used  the  DEGEM  system  for  mathematics  only,  while  the  Grade 
2B  (n=18)  class  used  the  system  for  language  arts  only.  As  can  be  seen,  the  2A 
class  is  significantly  stronger  than  the  2B  class  in  all  areas  regardless  of  the  subject 
for  which  the  students  were  receiving  CAI. 

The  Grade  3 A (n=22)  class  used  the  DEGEM  system  for  mathematics  only,  while 
the  Grade  3B  (n=16)  class  used  the  system  for  language  arts  only.  As  can  be 
observed,  there  is  a significant  difference  in  favor  of  the  3B  group  on  the  total 
language  score.  It  should  be  noted  that  this  group  also  tends  to  be  stronger  than  the 
other  on  all  measures,  although  the  differences  are  not  significant. 

The  results  of  the  Grade  4 (ii=22)  class,  which  only  used  the  system  for 
mathematics,  were  compared  to  the  results  of  the  Grade  5 (n=19)  class,  which  only 
used  the  system  for  language  arts.  No  significant  differences  were  noted.  The 
Grade  5 group  tended  to  be  slighdy  stronger  on  the  total  reading  and  spelling,  while 
the  Grade  4 group  tended  to  show  strengths  on  the  total  mathematics  and  on  the 
total  battery. 

The  results  from  the  Grade  5 class  (n=19),  a class  which  only  used  the  system  for 
language  arts,  were  compared  to  the  Grade  6 class  results  (n=19),  a class  which 
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only  used  the  system  for  mathematics.  A significant  difference  is  reported  for  the 
Grade  6 class  on  the  total  mathematics  and  on  the  total  battery.  The  same  class 
showed  itself  to  be  stronger  in  all  other  areas  although  these  results  were  not 
significant. 

Qualitative  data  regarding  student  achievement  were  collected  from  teachers  at  both 
sites  by  using  a questionnaire  and  a structured  interview.  In  1988  a total  of  eight 
teachers  from  Kehewin  responded,  while  a total  of  nine  teachers  from 
Onchaminahos  responded.  For  any  given  item,  however,  usually  about  fourteen  of 
the  possible  seventeen  teachers  would  respond.  Occasionally  the  number 
responding  was  as  low  as  eleven  or  as  high  as  sixteen.  These  figures  hold  for  any 
data  reported  from  the  questionnaire.  The  actual  numerical  values  of  the  questions 
for  this  section  are  reported  in  Table  2.  In  1989  nine  teachers  from  Onchaminahos 
and  two  teachers  from  Kehewin  responded  to  the  questionnaire.  Further,  in  1988- 
89  a selected  group  of  parents  (i.e.,  had  a child  using  DEGEM  and  was  part  of  the 
support  staff  at  the  school)  were  invited  to  participate  in  a face-to-face  interview  to 
report  on  the  effect  of  DEGEM  on  their  children.  These  results  will  be  reported  at 
the  end  of  this  section. 

In  1988,  on  the  student  achievement  portion  of  the  questionnaire,  thirteen  teachers 
responded  to  questions  regarding  mathematics  and  eight  teachers  responded  to 
questions  regarding  language  arts.  It  should  be  noted  that  at  Onchaminahos  School 
classes  use  DEGEM  for  either  mathematics  or  language  arts,  and  at  Kehewin  almost 
all  staff  used  it  for  mathematics  and  fewer  used  it  for  language  arts.  When  asked 
whether  DEGEM  contributed  to  the  improvement  in  basic  arithmetic  skills  (1988), 
both  groups  agreed,  with  Kehewin  being  more  affirming;  however,  in  1989  the 
Onchaminahos  teachers  showed  much  stronger  agreement  for  the  positive  effect  in 
arithmetic  than  did  either  group  in  1988.  With  respect  to  problem  solving  skills, 
again  there  was  general  agreement,  although  the  Onchaminahos  teachers  were  less 
positive  in  1989.  When  asked  about  improvement  on  various  dimensions  of 
language  arts  skills,  both  groups  responded  in  the  neutral  to  agree  range.  The  sole 
exception  was  in  vocabulary  skills  where  the  responses  were  in  the  neutral  to 
disagree  range.  In  1989,  there  was  a general  down  turn  in  the  Onchaminahos 
teachers'  responses  in  all  areas  dealing  with  language  arts  and  DEGEM's 
effectiveness.  Although  all  scores  remained  in  the  neutral  to  agree  range,  there  was 
a major  drop  in  teachers'  perceptions  in  the  area  of  reading  fluency.  In  1988,  the 
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administrators  indicated  they  had  noticed  an  overall  improvement  in  basic  arithmetic 
and  language  arts  skills  since  the  arrival  of  DEGEM. 

The  results  of  the  open-ended  questions  indicated  (1988)  that  teachers  believed  that 
DEGEM  had  contributed  the  most  to  their  students'  progress  by  providing  them 
with  drill  and  practice  in  a variety  of  areas.  Some  teachers  indicated  that  the  ten- 
minute  time  limit  tended  to  keep  students  more  on  task.  When  asked  for  one  way  to 
improve  the  system  the  answers  were  diverse  but  interesting.  One  teacher 
suggested  that  similar  worksheet  activities  be  made  available.  Other  comments 
were:  "Improve  the  re-inforcement  provided  to  the  students  for  correct  answers"; 
"Have  one  terminal  to  monitor  student  progress";  "Have  the  computer  provide  hints 
as  students  begin  to  have  problems";  "Match  the  computer  grade  levels  to  Alberta 
grade  levels,  the  present  ones  don't  correspond";  "Give  the  students  more  personal 
feedback  (reinforcement)". 

In  1988,  the  Steering  Committee  believed  that  the  impressions  of  the  community 
could  add  valuable  information  to  the  data  base.  Initially,  it  was  decided  to  survey  a 
representative  sample  of  parents  in  both  communities;  however,  after  meeting  with 
the  Steering  Committee  and  discussing  the  concept,  it  was  decided  that,  in  order  to 
respect  community  sensitivities,  a survey  of  a smaller  purposive  sample  would  be 
more  appropriate  at  this  time.  Therefore,  it  was  decided  that  in  order  to  participate 
in  the  survey,  a parent  must  have  a child  in  the  school  using  DEGEM,  and  the 
parent  must  be  part  of  the  paraprofessional  support  staff  at  the  school.  It  should  be 
noted  that  this  group  of  individuals  would  not  likely  be  representative  of  the 
community  at  large,  as  they  would  have  a more  informed  perception  than  a 
randomly  selected  group  would  have.  The  systems  manager  in  each  school 
arranged  a number  of  interviews  with  parents  in  his  own  school.  Originally, 
thirteen  parent  interviews  were  to  be  conducted;  however,  with  the  difficulties  in 
scheduling,  and  absenteeism  on  the  date  of  the  interviews,  the  final  sample  size  was 
seven:  four  at  Onchaminahos;  and  three  at  Kehewin.  A face-to-face  structured 
interview  was  conducted  with  each  parent.  The  seven  parents  represented 
seventeen  children  in  attendance  in  either  school.  The  first  question  asked  dealt 
with  awareness.  All  parents  indicated  that  they  were  aware  that  the  DEGEM  system 
was  being  used  in  their  school.  One  parent  indicated,  however,  that  it  would  be 
unlikely  he  would  have  known  of  the  system  had  he  not  been  working  in  the 
school. 


12 


Parents  were  then  asked  if  their  children  talked  about  DEGEM  at  home.  Four  of  the 
seven  parents  responded  in  the  affirmative.  One-half  of  those  responding  indicated 
that  their  children  "talked  a lot"  about  the  system.  Most  of  the  comments  made 
about  the  system  at  home  are  positive  in  nature  (e.g.,  "I  like  it";  "It  helps  me  learn  a 
lot.").  Only  two  parents  indicated  that  any  of  their  children  made  any  negative 
comments  at  home. 

When  asked  whether  they  thought  there  was  any  attitude  change  in  their  children 
since  using  the  system,  three  parents  responded  affirmatively,  (e.g.,  "They  look 
forward  to  school  and  the  computer";  "They  seem  to  know  a lot  more.").  One  of 
the  parents  who  had  seen  a positive  change  in  his  children  indicated  that  this  may 
not  be  a consequence  of  using  the  computer.  Four  parents  saw  no  change  in 
attitude.  Regarding  whether  they  noticed  any  change  in  marks,  four  parents  noticed 
a change,  especially  in  reading  comprehension,  mathematics,  and  spelling.  One 
parent  saw  no  change,  and  two  parents  said  it  was  difficult  to  determine  whether  the 
changes  could  be  attributed  to  DEGEM. 

When  asked  to  rate  the  impact  of  the  system  on  their  children,  all  responded 
positively.  As  well,  all  indicated  that  they  would  like  to  have  an  opportunity  to  use 
the  computer  themselves.  In  fact,  some  were  already  doing  so.  A sampling  of 
general  comments  follows  - "Not  enough  material  - sort  of  limited";  "Parents 
should  use  it";  "Makes  kids  lazier";  "Great  having  a computer  in  the  school"; 
"Access/feedback  right  away";  "Gave  students  a better  understanding". 
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Table  2 

Mean  Values  of  Student  Achievement  Questions  - Teacher  Questionnaire 


1988 

1989 

Keh 

One 

One 

12. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  basic  arithmetic  skills. 

3.2 

2.9 

4.3 

13. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  problem  solving  skills. 

3.2 

2.9 

2.4 

14. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  reading  comprehension  skills. 

3.1 

2.8 

2.7 

15. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  vocabulary  skills. 

3.1 

2.8 

2.6 

16. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  decoding  skills. 

3.6 

3.3 

2.7 

17. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  reading  fluency. 

3.5 

3.0 

2.1 

18. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  grammar  skills. 

3.5 

3.1 

2.9 

19. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  punctuation  skills. 

3.6 

3.1 

3.1 

20. 

The  DEGEM  system  has  contributed  to  my 
students'  overall  spelling  skills. 

3.1 

2.9 

2.4 

DISCUSSION:  Question  One 

The  primary  site  for  the  quantitative  data  collection  was  Onchaminahos  School; 
qualitative  data  was  collected  from  both  sites.  In  1988  the  teachers  at  both  schools 
perceived  DEGEM  to  have  assisted  in  the  area  of  student  achievement  with  more 
effect  seen  in  the  mathematics  area.  The  administrators  were  generally  in  agreement 
with  the  teaching  staff  although  they  perceived  the  system  as  having  a greater  effect 
than  did  the  teachers.  There  was  a general  drop  in  perceived  effectiveness  of  the 
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DEGEM  system  in  language  arts  at  Onchaminahos  in  1989.  There  was,  however, 
an  equally  large  rise  in  the  perceived  effectiveness  in  the  arithmetic  area  by  the  same 
group.  The  quantitative  data  that  were  collected  and  examined  did  not  allow  for  any 
conclusions  to  be  drawn.  However,  two  general  statements  can  be  made.  First,  in 
1988,  the  distribution  of  the  students  in  their  classes  may  not  have  been  random,  in 
the  statistical  sense;  the  students  in  the  classes  may  not  have  been  divided  equally 
according  to  ability;  however,  it  is  apparent  that  some  classes  are  much  stronger  as 
a group  (e.g..  Grade  2A;  Grade  6).  The  total  impact  of  this  variation  is  unknown. 
For  example,  if  the  more  able  students  (or  the  less  able  students)  derive  greater 
benefit  from  DEGEM  there  would  be  an  unequal  weighting  on  the  final  result  which 
could  lead  to  erroneous  conclusions.  Second,  the  design  assumes  that  the  gain  in 
instruction  will  be  limited  to  the  subject  to  which  the  students  are  assigned.  For 
example,  it  is  assumed  that  if  a student  only  uses  the  computer  for  mathematics  that 
this  will  have  no  effect  on  his  language  arts.  This  appears  to  have  face  validity; 
however,  it  would  have  been  better  to  match  the  class  receiving  CAI  in  mathematics 
to  a class  receiving  classroom  instruction  alone;  however,  this  was  not  possible 
given  the  initial  low  student  numbers.  There  is  an  unknown  factor  introduced  by 
having  all  students  participate  in  CAI  for  one  subject  or  the  other.  These  two 
factors  may  have  some  impact  on  the  data;  however,  the  degree  of  impact  will 
remain  unknown.  At  this  point,  with  the  useable  data  collected  (1988-1989),  no 
significant  differences  between  groups  can  be  attributed  to  CAI  on  the  DEGEM 
system.  Parent  interviews  presented  a mixed  reaction  to  the  system;  however,  the 
general  indication  was  that  the  parent  population  selected  was  likely  more  aware  of 
the  system  and  its  strengths  and  weaknesses  than  those  parents  in  die  community  at 
large.  The  teachers'  reactions  to  the  effectiveness  of  the  DEGEM  system  are 
discussed  fully  in  question  six. 
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Question  Two  - How  appropriate  is  the  DEGEM  CAI  courseware  according  to 
the  Alberta  Education  Clearinghouse  criteria  and  standards,  including  management 
systems,  content  sequence  and  range,  and  instructional  format? 

The  results  of  the  study  which  pertain  to  this  question,  as  prepared  by  Alberta 
Education,  are  found  in  Appendix  C. 
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Question  Three  - How  appropriate  is  the  DEGEM  on-board  diagnostic  test 
capability  compared  to  the  standardized  diagnostic  tests  chosen  as  test  instruments 
in  the  present  evaluation? 

These  data  will  be  reported  using  three  case  studies.  For  the  first  two,  each  student 
was  selected  at  random  from  the  DEGEM  class  lists  which  were  sorted  by  urgency. 
That  is,  the  system  has  identified  these  students  as  requiring  assistance.  One 
student  was  selected  from  language  arts  and  one  from  mathematics.  A diagnostic 
report  was  written  for  each  child  based  on  the  DEGEM  results.  The  CAT  results 
for  the  same  students  were  analyzed  and  a diagnostic  profile  was  written  from  the 
perspective  of  the  CAT  information.  The  two  reports  (i.e.,  DEGEM  and  CAT) 
were  then  compared.  The  third  student  was  identified  as  having  difficulty 
according  to  the  CAT  results  and  was  selected  at  random.  The  procedure  outlined 
above  to  analyze  the  results  was  reversed.  Copies  of  the  DEGEM  printouts  and 
CAT  results  can  be  found  in  Appendix  B. 

CASE  A 

The  first  student  (C.J.),  is  a nine-year-old  female  in  grade  four  at  Kehewin  School. 
Her  name  was  selected  from  the  arithmetic  class  printout  dated  May  19,  1988, 
sorted  by  urgency.  Her  name  appeared  fifth  from  the  top  indicating  that  according 
to  the  DEGEM  system  she  was  one  of  the  students  requiring  teacher  assistance. 
The  printout  indicated  that  she  had  completed  her  sbcty-ninth  session  and  was  given 
an  overall  grade  equivalent  score  of  2.9  years  (Class  Average  3.4).  Eight  topics 
are  open  to  her  and  seven  remain  closed  as  her  grade  equivalent  score  is  too  low  to 
permit  her  access  to  these  topics.  She  has  been  dropped  a level  (more  than  90 
percent  of  the  questions  presented  were  responded  to  incorrectly,  or  the  first  three 
questions  were  incorrect)  in  four  of  the  eight  topics  open  to  her.  During  her  last 
session  she  was  presented  with  forty-one  questions  of  which  she  answered  twenty 
correctly.  Of  the  twenty-one  she  answered  incorrectly,  two  were  time  out  errors. 
That  is,  she  did  not  respond  within  the  allotted  time. 

To  determine  her  grade-equivalent  score  in  any  given  DEGEM  topic,  the  two-digit 
number  must  be  converted  to  years  and  months  (i.e.,  28  becomes  2.8  or  2 years  8 
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months).  For  Topic  2 the  printout  (25^1)  indicates  she  is  working  at  a Grade  2.5 
level.  She  has  been  dropped  a level  and  is  currently  working  at  Level  25.1.  To 
interpret  that,  the  Mathematics/Arithmetic  Exercise  Summary  book  is  used.  Under 
Topic  2 refer  to  Section  25.1.  This  section  indicates  that  C.J.  is  currently  working 
on  single  digit,  horizontal  addition  questions  like  6 + 8 = ? or  14  - 8 = ?.  Due  to  a 
drop  in  level,  DEGEM  indicates  that  she  needs  attention  in  the  following  areas: 

Topic  3 (25^1)  Vertical  Addition,  one-digit  to  two-digit  numbers,  for 
example,  53 

±4 

Topic  4 (27^1)  Vertical  Subtraction,  multiples  of  10  to  90,  for  example, 

80 

-M 

Topic  5 (33M)  Single  Digit  Multiplication,  for  example,  4x4  = ?;  and 
division  of  a two-digit  number  by  a one-digit  number,  for 
example  16/4  = ?. 

No  other  suggestions  are  given  as  to  what  course  of  action  the  teacher  should  take 
at  this  time.  In  Topic  5,  for  example,  it  is  not  known  whether  C.J.  is  having 
difficulty  with  multiplication,  or  division,  or  both.  A diagnostic  summary  based  on 
the  DEGEM  results  reads  as  follows:  C.J.  is  a grade  four  student  who  is  currently 
experiencing  a great  deal  of  difficulty  in  mathematics.  She  is  currently  functioning 
at  a Grade  2.9  level  (Class  Average  3.4).  She  is  having  difficulty  with  the  four 
basic  mathematics  operations.  During  her  last  lesson  she  was  dropped  a level  in 
four  topics  suggesting  she  had  made  a large  number  of  errors  or  perhaps  she  was 
experiencing  some  frustration  with  the  work.  The  DEGEM  information  indicates 
that  she  requires  additional  practice  with  her  basic  facts;  first  in  addition, 
subtraction,  and  multiplication,  then  division.  At  this  point,  she  is  working 
primarily  with  one-digit  and  two-digit  numbers.  However,  she  is  still  not  able  to 
work  with  two  two-digit  numbers,  unless  they  are  multiples  of  ten.  The  DEGEM 
results  suggest  that  C.J.  should  be  given  additional  drill  in  her  basic  computation 
facts.  Neither  the  printout  nor  the  summary  book  offer  any  explanation  or  specific 
remediation  activities  for  her  difficulties. 
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CJ.  wrote  the  Level  14  Canadian  Achievement  Test  in  May  1988  (Grade  Level 
4.8).  Her  grade-equivalent  score  on  the  total  battery  was  3.8.  For  the  purposes  of 
this  exercise,  only  the  mathematics  section  of  the  CAT  will  be  discussed.  The  CAT 
results  for  C.J.  are  reported  in  Table  3: 


Table  3 

C.J.'s  Canadian  Achievement  Test  Results  in  Mathematics 


Area 

Grade 

Stanine 

Percentile 

Class 

Equivalent 

Average 

Math  Computation 

4.0 

4 

25 

4.5 

Concepts  & Application 

5.3 

6 

61 

3.9 

Total  Math 

4.5 

5 

44 

4.2 

An  item  analysis  of  C.J.'s  CAT  results  indicates  difficulty  in  the  following  areas: 

Addition  - adding  four  four-digit  numbers 

- adding  columns  of  up  to  four-digit  numbers 

- horizontal  addition  of  up  to  four-digit  numbers 

- adding  like  fractions 

- adding  fractions,  horizontally,  no  regrouping 

Subtraction  - subtracting  three-digit  and  two-digit  numbers,  no 

regrouping 

- subtracting  three-digit  and  two-digit  numbers 

- subtracting  four-digit  and  three-digit  numbers 

- subtracting  two  five  digit  numbers 

- subtracting  like  fractions 

- subtracting  decimal  fractions 
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Multiplication  - multiplying  one-digit  numbers  by  ten,  horizontally 

- multiplying  a two-digit  number  by  a one-digit  number 

- multiplying  two  two-digit  numbers 

Division  - difficulty  in  all  areas,  however  she  can  divide  a two-digit 

number  by  a one-digit  number  (no  regrouping). 

As  her  scores  were  above  average  on  the  concepts  and  applications  section,  no 
recommendations  are  necessary.  The  CAT  results  indicate  that  C.J.  is  having 
difficulty  in  the  area  of  computation  skills  with  two-,  three-,  and  four-digit 
numbers.  Remediation  would  begin  in  the  addition  and  subtraction  areas  with 
particular  attention  to  number  columns  and  regrouping.  In  the  area  of 
multiplication,  C.J.  would  begin  multiplying  one-digit  numbers  by  multiples  of  ten 
and  then  by  other  two-digit  numbers.  Instruction  in  division  would  not  begin  until 
C.J.  has  mastered  the  skills  in  the  other  basic  operations. 

Specific  suggestions,  given  by  item,  are  available  in  the  Class  Management  Guide. 
For  example,  according  to  the  item  analysis  on  the  CAT,  C.J.  answered  question 
21  incorrectly.  This  question  is  under  Objective  45,  addition.  Suggestions  for  the 
intermediate  level  (grades  4-6)  are  found  on  page  1 10  of  the  Class  Management 
Guide.  One  suggestion  given  is  to  duplicate  a puzzle  found  on  the  page  with  the 
numbers  omitted.  C.J.  would  then  put  the  numbers  from  one  to  nine  in  empty 
circles  so  that  each  set  of  three  numbers  connected  by  lines  adds  up  to  15.  Another 
suggestion  recommends  the  use  of  a "Magic  Square",  basically  a grid  in  which 
some  numbers  are  missing  and  C.J.  would  have  to  fill  in  the  missing  numbers, 
ensuring  that  the  sum  of  each  row,  column,  and  diagonal  is  the  same  number. 

The  same  procedure  could  be  applied  for  any  incorrect  response  on  any  question 
where  the  total  number  of  errors  in  the  section  exceeded  50  percent  (designated  by 
an  "1"  on  the  CAT).  The  resultant  product  would  be  developed  into  a specific 
educational  plan  for  C.J.  It  should  also  be  noted  that  it  is  not  necessary  to  go  to  the 
level  of  item  analysis  to  obtain  this  type  of  information. 

When  the  CAT  and  DEGEM  results  are  compared,  C.J.  is  found  to  be  weak  in  the 
area  of  mathematical  computations.  However,  DEGEM  identifies  her  as  much 
weaker,  indicating  that  she  is  having  difficulty  with  basic  number  facts  and  simple 
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mathematical  operations.  On  the  CAT  she  was  able  to  obtain  correct  answers  on 
those  types  of  questions.  The  CAT  indicated  her  areas  of  weakness  were  at  a 
higher  level,  that  her  basic  facts  had  been  mastered,  and  her  difficulties  involved 
operations  with  two-,  three-  and  four-digit  numbers.  Therefore,  in  this  particular 
instance  there  is  some  agreement  as  to  the  areas  of  concern.  However,  there  does 
not  appear  to  be  a good  match  between  these  two  diagnostic  systems  with  respect  to 
the  level  that  C.J.  should  be  working  on  According  to  the  CAT,  C.J.  has  acquired 
skills  DEGEM  indicates  she  has  not  yet  mastered. 

Cases 

R.G.  is  a 12-year-old  male  in  grade  five  at  Onchaminahos  School  in  Saddle  Lake. 
His  name  was  selected  from  the  English  class  printout  of  May  1988,  sorted  by 
urgency.  His  name  appeared  at  the  top  of  the  list,  indicating  that  he  was  the  student 
most  urgently  requiring  teacher  assistance.  It  should  be  noted  that  the  criteria  to  be 
considered  "urgent"  does  not  depend  on  the  relative  grade  level  of  functioning  (i.e., 
the  student  the  furthest  behind  is  not  necessarily  the  most  "urgent").  Rather  it 
depends  on  the  number  of  areas  or  topics  in  which  a student  is  "stuck",  or  has  been 
dropped  a level.  The  printout  indicated  that  he  had  completed  his  fifteenth  session 
and  was  given  an  overall  grade-equivalent  score  of  3.7  years  (Class  Average  3.3). 
Thirteen  topics  are  open  to  him  and  two  remain  closed  as  he  is  below  the  required 
entry  level  for  these  topics.  The  printout  indicates  that  R.G.  is  having  difficulty  in 
the  first  group  of  exercises  for  topics  3,  8,  and  15;  in  the  second  group  of  exercises 
for  topic  10,  and  in  the  third  group  of  exercises  for  topics  1,  4,  5,  9,  11,  12,  13, 
and  14.  There  is  only  one  topic,  topic  2,  in  which  he  is  not  experiencing  any 
difficulty  at  this  time.  During  his  last  session  he  was  presented  with  forty-five 
questions  of  which  he  answered  twenty  correctly.  Of  the  twenty-five  he  answered 
incorrectly,  there  were  no  time  out  errors.  That  is,  he  consistently  responded 
within  the  allotted  time. 

The  printout  and  the  English  Course  Teacher’s  Manual  indicate  that  R.G.  is  having 
difficulty  with  the  exercises  in  the  following  areas: 

Topic  1 (>36)  Letter  and  Word  Attack  Skills  - He  is  having  a problem  with 

the  third  set  of  exercises  involving  different  consonant 
sounds,  for  example  c (="s",  ="k"). 


21 


Topic  3 (-37) 
Topic  4 (>37) 

Topic  5 (>36) 
Topic  8 (-37) 

Topic  9 (>37) 

Topic  10  (=37) 

Topic  11  (>36) 

Topic  12  (>36) 

Topic  13  (>36) 
Topic  14  (>36) 
Topic  15  (-38) 


Pronouns  and  Noun  Endings  - He  is  having  problems  with 
the  first  set  of  exercises  involving  the  plural  of  nouns  ending 
in  "y". 

Modals  and  Auxiliary  Sentence  Patterns  - He  is  having 
difficulty  with  the  third  set  of  exercises  involving  the  correct 
positioning  and  use  of  auxiliaries  like  "going  to"  as  future 
before  the  main  verb  in  a sentence. 

Verb  Forms,  Verbs  - He  is  having  problems  with  the  third 
set  of  exercises  involving  the  proper  tense  for  the  verb  "be". 
Noun  Phrases,  Prepositions,  Phrasal  Verbs  - R.G.  is  having 
problems  with  the  first  set  of  exercises  involving 
prepositions  of  time  "on,  at,  in". 

Sentence  Patterns,  Evaluative  Reading  - He  is  having 
difficulty  with  the  third  set  of  exercises  involving  the  use  of 
future  tense  with  "going  to"  in  an  affirmative  sentence. 

Verb  Usage,  Detailed  Information  - He  is  having  some 
problems  with  the  second  set  of  exercises  involving  the  use 
of  past  irregular  tense  with  time  expressions. 

Coordination,  Reference  - R.G.  is  having  difficulty  with  the 
third  set  of  exercises  involving  pronoun  reference, 
specifically,  the  pronoun  "it"  as  a subject  or  object. 
Punctuation,  Classification,  Context  Clues  - He  is  having 
difficulty  with  the  third  set  of  exercises  involving 
classification  (i.e.  matching  class  names  to  examples  of  that 
class). 

Detail  and  Main  Idea  - This  indicates  that  the  student  is 
having  difficulty  with  the  third  set  of  exercises  involving  the 
recall  of  detail  from  a three- sentence  passage. 

Sequence  - R.G.  is  having  some  problems  with  the  third  set 
of  exercises  involving  understanding  the  sequence  in  a 
compound  sentence. 

Cause  and  Effect,  Conclusions,  Inferences  - This  indicates 
that  the  student  is  having  difficulty  with  the  first  set  of 
exercises  requiring  him  to  infer  the  effect  on  the  basis  of 
information  given  in  one  sentence. 
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A diagnostic  summary  based  on  the  DEGEM  results  reads  as  follows:  R.G.  is  a 
grade  five  student  who  is  currently  experiencing  difficulty  in  the  reading  and 
language  areas.  He  is  currently  functioning  at  the  Grade  3.7  level  (Class  Average 
3.3).  He  is  having  difficulty  with  word  attack  skills,  grammar,  syntax, 
classification,  and  reading  comprehension  skills  (including  recall  of  detail, 
sequencing,  inference  and  determining  cause  and  effect).  During  his  last  lesson,  he 
experienced  difficulty  in  11  of  the  15  topics,  suggesting  that  he  is  making  a large 
number  of  errors  and  may  be  experiencing  some  frustration.  The  DEGEM 
information  indicates  that  he  should  be  receiving  remediation  in  three  major  areas: 
word  attack  skills;  grammar  and  syntax;  and  reading  comprehension  skills. 
Particular  attention  should  be  paid  to  differing  consonant  sounds,  verbs  and  verb 
tenses,  parts  of  speech  (i.e.,  pronouns,  prepositions,  and  auxiliaries),  recalling 
detail,  sequencing,  and  inferring  cause  and  effect.  The  DEGEM  results  suggest  that 
R.G.  should  be  working  on  some  relatively  basic  decoding,  reading 
comprehension,  language  expression,  and  language  mechanics  skills.  It  offers  no 
further  explanation  or  specific  remediation  activities  for  his  difficulties. 

R.G.  wrote  the  Level  15  Canadian  Achievement  Test  in  May  1988  (Grade  Level 
5.8).  His  total  battery  score  was  Grade  5.8.  For  the  purposes  of  this  exercise,  the 
reading,  spelling,  and  language  sections  of  the  CAT  will  be  discussed.  The  CAT 
results  for  R.G.  are  reported  in  Table  4: 


Table  4 

R.G.'s  Canadian  Achievement  Test  Results  in  Language 


Area  Grade  Equivalent 

Stanine 

Percentile 

Class  Average 

Reading  Vocabulary 

5.2 

5 

41 

3.2 

Reading  Comprehension 

7.8 

6 

64 

3.9 

Total  Reading 

6.2 

5 

55 

3.5 

Spelling 

7.0 

6 

62 

5.6 

Language  Mechanics 

11.8 

7 

82 

4.0 

Language  Expression 

3.4 

3 

21 

3.4 

Total  Language 

5.1 

5 

44 

3.7 
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CAT  indicate  difficulty  in  the  following  areas: 

- Some  weakness  with  same  meaning 

- Some  weakness  with  contextual  analysis 

- Weak  critical  comprehension 

- Some  difficulty  with  silent  single  letters  and  "r" 
control 

- Some  problems  with  punctuation  (i.e., 
exclamation  marks  and  commas  in  addresses 
and  quotations) 

- Problems  with  pronouns  (him,  me) 

- Problems  with  verbs  (present  tense  and  perfect 
infinitive  tense) 

- Problems  with  superlative  adjectives 

- Problems  with  subjects/verbs 

- Problems  with  paragraph  organization  (i.e. 
irrelevant  sentences). 

The  CAT  results  indicate  that  R.G.’s  major  area  of  difficulty  is  language 
expression,  primarily  in  the  areas  of  grammar,  sentence  structure,  and  paragraph 
organization.  Additional  remediation  should  be  given  in  the  areas  of  language 
mechanics  (i.e.  punctuation)  and  reading  comprehension  (i.e.  contextual  analysis 
and  critical  comprehension,  both  involving  inference).  If  desired,  some 
remediation  could  be  introduced  in  spelling  with  silent  single  letters  and  "r"  control. 
Specific  suggestions,  given  by  item,  are  available  in  the  Class  Management  Guide. 
The  suggestions  are  divided  by  level  (i.e.,  elementary,  intermediate,  junior  high) 
and  will  not  be  discussed  as  the  format  is  the  same  as  those  listed  above  in  CASE  A. 

When  the  CAT  and  DEGEM  results  are  compared,  R.G.  is  found  to  be  weak  in  the 
area  of  language  expression.  Both  diagnostic  systems  indicate  that  further  work  is 
required  in  the  areas  of  grammar,  specifically  verbs,  verb  tense,  parts  of  speech, 
and  subject/verb  agreement.  Both  systems  suggest  that  R.G.  may  have  some 
weaknesses  in  the  area  of  organization.  DEGEM  suggests  that  he  has  difficulties 
sequencing,  and  the  CAT  indicates  he  has  a problem  with  sentence  structure  and 
paragraph  organization.  However,  the  systems  provide  conflicting  data  regarding 
reading  comprehension.  DEGEM  suggests  that  he  requires  assistance  with  some 
basic  comprehension  skills  (i.e.,  recalling  detail,  sequencing,  and  determining 


An  item  analysis  of  R.G.'s 
Reading  Vocabulary 
Reading  Comprehension 

Spelling 

Language  Mechanics 
Language  Expression 
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cause  and  effect)  while  the  CAT  indicates  that  he  has  mastered  these  skills  and 
suggests  remediation  at  a higher  level  of  skill  involving  contextual  analysis  and 
critical  comprehension.  Therefore,  there  appears  to  be  some  match  between  these 
two  diagnostic  systems  in  the  area  of  grammar  and  organization;  however,  in  the 
area  of  reading  comprehension  the  CAT  results  indicate  he  is  able  to  function  in 
areas  that  DEGEM  indicates  he  has  not  mastered.  Further,  the  CAT  is  suggesting 
remediation  at  higher  levels.  Finally,  it  may  be  noteworthy  that  according  to  both 
measures  R.G.  is  generally  functioning  above  his  class  average.  The  DEGEM 
system  identifies  12  areas  where  he  requires  assistance  while  the  CAT  identifies 
one,  possibly  two.  In  addition,  the  CAT  identifies  four  areas  of  strength. 

Case  C 

C.G.,  is  a 10-year-old  male  in  grade  three  at  Kehewin  School.  He  wrote  the  Level 
13  Canadian  Achievement  Test  in  May  1988  (Grade  Level  3.8).  His  total  battery 
score  yielded  a grade-equivalent  of  2.6.  The  CAT  indicated  that  C.G.  was  low  in 
all  areas  of  mathematics;  therefore,  for  the  purposes  herein,  only  the  mathematics 
section  of  the  CAT  will  be  discussed.  The  CAT  results  for  C.G.  are  reported  in 
Table  5. 


Table  5 

C.G.'s  Canadian  Achievement  Test  Results  in  Mathematics 


Area  Grade  Equivalent 

Stanine 

Percentile 

Class  Average 

Math  Computation  2.9 

3 

11 

3.4 

Concepts  & Application  2.5 

3 

14 

3.3 

Total  Math  2.6 

2 

10 

3.4 

An  item  analysis  of  C.G.’s  CAT  indicates  difficulty  in  the  following  areas: 
Addition  - adding  2 two-digit  numbers,  horizontally 

- adding  3 two-digit  numbers,  horizontally 

- adding  3 two-digit  numbers 

- adding  2 two-digit  and  2 one-digit  numbers 
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Subtraction 

Multiplication 

Division 

Numeration 

Number  Theory 
Number  Properties 

Common  Scales 

Measurement 

Problem  Solving 


- adding  2 four-digit  numbers 

- adding  up  to  three-digits,  horizontally 

- adding  4 four-digit  numbers 

- subtracting  two  and  one-digit  numbers 

- subtracting  2 two-digit  numbers 

- subtracting  three  and  two-digit  numbers 

- basic  facts,  2 one-digit  numbers 

- multiplying  10  by  a one-digit  number 

- multiplying  two  by  one-digit  numbers,  no  regrouping 

- multiplying  three  by  one-digit  numbers,  horizontally 

- basic  facts,  2 one-digit  numbers 

- basic  facts,  two-digit  divided  by  one-digit  numbers 

- multiples  of  10  divided  by  one-digit  numbers 

- renaming  thousands,  hundreds,  and  ones 

- adding  number  line  sentences 

- rounding  three-digit  numbers,  nearest  100 

- odd,  even  numbers 

- factors  of  4 and  8 

- identifying  elements  of  multiplication 

- division  equations,  solving  for  the  box 

- symbol  for  true  sentences,  multiplication 

- reading  clock  and  adding  time 

- counting  money 

- reading  calendar,  day  on  date 

- perimeter  of  rectangle 

- story  problem,  multiplying  kg 

- story  problem,  convert  m to  cm 

- estimate  volume,  L 

- identifying  process,  subtraction 

- identifying  process,  subtraction  and  addition 

- solving  by  multiplying 

- solving  by  dividing 


The  CAT  also  identified  some  weaknesses  in  the  areas  of  graphs  and  geometry. 
These  areas  will  not  be  discussed  here  as  they  are  not  included  in  the  DEGEM 
diagnostic  system. 
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The  CAT  results  indicate  that  C.G.  is  having  difficulty  in  all  areas  of  mathematics 
computation;  some  areas  of  numeration,  number  theory  and  number  properties; 
some  areas  of  measurement  and  common  scales;  and  in  problem  solving.  Although 
he  knows  his  basic  addition  and  subtraction  facts  and  understands  the  commutative 
and  associative  properties,  he  requires  further  remediation  before  he  is  ready  to 
work  with  larger  numbers  (i.e.,  three-  and  four-digit)  using  these  operations.  He 
requires  drill  in  his  basic  facts  for  both  multiplication  and  division.  His  work  in 
problem  solving  should  reflect  this  by  supporting  his  strengths  in  addition  and 
subtraction  and  should  require  little  or  no  work  in  multiplication  or  division  at  this 
time.  In  addition,  emphasis  should  be  placed  on  improving  his  knowledge  of  and 
skills  in  measurement,  and  using  common  scales  (i.e.,  money,  calendar,  clock). 
Specific  suggestions,  given  by  item,  are  available  in  the  Class  Management  Guide. 

C.G.'s  name  appeared  third  from  the  top  on  the  DEGEM  arithmetic  printout  for  her 
class,  dated  May  1988,  sorted  by  urgency.  This  indicates  that  according  to  the 
DEGEM  system  he  is  one  of  the  students  requiring  teacher  assistance.  The  printout 
indicated  that  he  had  completed  his  seventy-first  session  and  was  given  an  overall 
grade  equivalent  of  2.6  years  (Class  Average  2.9).  Seven  topics  are  open,  that  is, 
he  has  access  to  these  topics.  The  teacher  has  closed  the  remaining  topics  (2)  to  all 
of  her  students.  C.G.  has  been  dropped  a level  (more  than  90  percent  of  the 
questions  presented  were  responded  to  incorrectly,  or  the  first  three  questions  were 
incorrect)  in  four  of  the  seven  topics  open  to  him.  During  his  last  session  he  was 
presented  with  twenty-five  questions  of  which  he  answered  nineteen  correctly.  Of 
the  six  he  answered  incorrectly,  one  was  a time  out  error.  That  is,  he  did  not 
respond  within  the  allotted  time. 

For  Topic  1 the  printout  (25^2)  indicates  he  is  working  at  a grade  2.5  level.  He  has 
been  dropped  a level  and  is  currently  working  at  Level  25.1.  To  interpret  that,  the 
Mathematics/Arithmetic  Exercise  Summary  Manual  is  used.  Under  Topic  1 refer  to 
Section  25.1.  This  section  indicates  that  C.G.  is  currently  working  on  numerical 
systems,  specifically,  filling  in  the  missing  number  (with  increasing  even  numbers, 
from  2 to  24).  Other  areas  that  DEGEM  indicates  he  needs  attention  due  to  this 
drop  in  level  are: 
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Topic  3 (27^1)  Vertical  Addition,  three  addends:  multiples  of  10;  sum  up  to 
90. 

Topic  4 (26^1)  Vertical  Subtraction,  minuends  of  20  to  49  and  subtrahends  of 
1 to  9. 

Topic  14  (22^08)  Word  Problems,  single  digit  addition  and  subtraction. 

There  are  no  further  suggestions  as  to  what  course  of  action  the  teacher  should  take 
at  this  time.  A diagnostic  summary  based  on  the  DEGEM  results  would  be  as 
follows:  C.G.  is  a grade  three  student  who  is  currently  experiencing  difficulty  in 
mathematics.  He  is  currently  functioning  at  a Grade  2.6  level.  He  is  having 
difficulty  with  the  operations  of  addition  and  subtraction.  During  his  last  lesson,  he 
was  dropped  a level  in  four  topics  which  would  normally  suggest  that  he  made  a 
number  of  errors;  however,  in  this  particular  case,  C.G.  only  made  six  errors  out  of 
twenty-five  questions.  At  this  point  in  time,  he  would  be  working  with  simple 
addition  and  subtraction  operations,  with  primarily  one-  and  two-digit  numbers.  In 
addition,  some  work  in  numerical  systems  is  required  to  enhance  his  understanding 
of  number  theory. 

When  the  CAT  and  DEGEM  are  compared,  C.G.  is  found  to  be  weak  in  the  area  of 
mathematics  computation  with  DEGEM  placing  him  slightly  weaker  (0.3).  Both 
diagnostic  systems  specify  remediation  in  the  operations  of  addition  and  subtraction 
using  one-  and  two-digit  numbers.  However,  the  CAT  diagnostics  offer  more 
detail  and  include  additional  areas  (i.e.,  number  theory,  measurement,  and  common 
scales). 

DISCUSSION:  Question  THREE 

There  is  no  clear  and  simple  answer  to  this  question.  The  pivotal  point  of  this 
discussion  is  How  can  the  DEGEM  results  be  accurately  compared  to  any  other 
system  or  standardized  instrument?.  Basically,  DEGEM  results  yield  raw  scores 
and  grade  equivalents.  The  comparison  of  raw  scores  is  not  appropriate  and  the 
meaning  of  grade  equivalent  scores  vary  from  instrument  to  instrument.  As  far  as 
can  be  determined,  as  it  is  never  stated  clearly  in  any  of  the  literature  or  manuals 
which  come  with  the  system,  the  DEGEM  grade  equivalent  scores  appear  to  be 
criterion  referenced  scores.  That  is,  a curriculum  was  analyzed  (probably  for  the 
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state  of  New  York),  and  it  was  determined  that  a child  in  the  fourth  month  of  grade 
three  was  able  to  do  a specific  type  of  problem.  Therefore,  when  a child  is 
successful  at  similar  problems  on  DEGEM  he  is  given  a grade  equivalent  score  of 
3.4.  On  the  CAT,  which  uses  Canadian  norms,  if  a grade  two  student  were  to 
obtain  a grade  equivalent  score  of  3.4,  it  means  that  the  student  obtained  the  same 
raw  score  as  a student  in  Grade  3.4  would  have  obtained  on  the  level  12  test.  It 
does  not  necessarily  mean  the  student  is  functioning  at  the  grade  3.4  level.  It  is 
apparent  that  comparisons  cannot  be  made  by  grade  equivalent  scores. 
Subsequently,  the  case  study  format  was  decided  upon.  Before  leaving  this  issue  it 
should  be  stated  that  it  will  be  important  for  Alberta  Education,  as  they  attempt  to 
answer  Question  Two,  to  examine  the  issue  of  how  the  DEGEM  system  is  normed 
(or  referenced)  and  what  the  implications  are  for  Alberta  students. 

As  was  seen  in  the  results  section,  there  are  some  advantages  and  disadvantages  to 
the  DEGEM  diagnostic  system.  The  intent  of  this  discussion  is  to  highlight  these 
with  respect  to  the  capabilities  of  the  chosen  standardized  diagnostic  test  instrument, 
the  Canadian  Achievement  Test.  The  discussion  will  revolve  around  three  central 
issues:  the  validity  and  accuracy  of  the  diagnostic  assessment  itself;  the  utility  of  the 
results  for  developing  an  individualized  program  for  the  student;  and  its  ease  of  use 
for  the  teacher. 

With  respect  to  the  first  issue  regarding  the  validity  and  accuracy  of  the  DEGEM 
diagnostic  assessment  itself,  it  is  important  to  examine  the  results  of  the  DEGEM 
printout  in  context.  In  other  words,  it  is  important  to  consider  the  results  of  the 
DEGEM  printout  in  conjunction  with  the  student’s  daily  work,  his  school  marks, 
his  past  academic  history,  and  his  teacher's  perceptions,  as  diagnostic  instruments 
are  not  generally  intended  to  be  used  in  isolation.  Since  this  level  of  detailed 
information  is  not  available  for  each  student,  teachers  were  asked  how  the  DEGEM 
results  compared  to  other  information  they  had  on  their  students.  There  was 
general  agreement  that  the  system  provided  information  which  was  similar  to  the 
other  information  they  had;  however,  with  respect  to  the  issue  of  grade  equivalents, 
there  was  general  dissatisfaction  with  the  grade  equivalent  scores  (i.e.,  they  seem  to 
the  teachers  to  be  too  low).  It  is  noteworthy  that  the  grade  equivalent  scores 
continue  to  be  the  most  used  "diagnostic"  feature  of  the  printout.  Perhaps  this  is 
because  teachers  recognize  these  scores,  even  though  the  true  meaning  may  be 
rather  obscure. 
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As  the  case  studies  demonstrate  when  the  information  received  from  the  DEGEM 
printout  was  compared  with  information  obtained  from  the  CAT,  there  is  some 
match  between  these  two  diagnostic  systems.  However,  in  all  three  cases,  DEGEM 
appears  to  place  the  student  at  a lower  grade  equivalent  level  than  the  CAT. 
DEGEM  also  appears  to  focus  more  on  the  skills  involved  in  the  subject  matter 
rather  than  on  the  concepts.  This  is  true  for  mathematics;  of  the  fifteen  topics 
covered,  only  two  or  three  topics  could  be  considered  to  refer  to  concepts.  This  is 
also  true  for  language  arts  where,  again,  only  about  five  of  the  fifteen  topics  involve 
concepts.  The  remaining  topics  heavily  emphasize  skills  in  language  mechanics  and 
expression.  The  CAT  results  offer  a more  balanced  weighting,  placing  a more 
equitable  emphasis  on  both  concepts  and  skills.  This  difference  may  be  a reflection 
of  the  nature  of  a CAI  system  which  is  better  suited  to  provide  support  for  the  basic 
skills  involved  in  the  mathematics  and  language  arts  curricula.  Depending  upon  the 
specific  needs  of  the  student  and  the  teacher,  this  may  be  perceived  as  a strength  or 
a limitation. 

With  reference  to  the  second  issue,  the  utility  of  the  DEGEM  diagnostic  system 
results  in  the  development  of  individualized  programs  for  students,  it  is  important  to 
consider  both  the  student's  strengths  and  weaknesses.  If  a teacher  knows  what  a 
child  does  well,  this  information  can  be  used  as  a starting  point  to  plan  the  child's 
program;  at  this  point  success  and  motivation  are  automatically  built  in.  One 
drawback  of  the  DEGEM  system  results  is  that  it  only  reports  weaknesses  (i.e., 
when  a student  is  dropped  a level  or  is  "stuck").  In  order  to  discover  a student's 
strengths  the  teacher  is  required  to  use  the  grade  equivalent  score  for  a topic,  refer 
to  the  Mathematics/Arithmetic  Exercise  Summary  Manual  and/or  the  English 
Exercise  Description,  examine  what  specific  types  of  exercises  the  student  is 
working  on,  and  which  skills  the  student  has  previously  mastered.  In  comparison, 
the  CAT  offers  a one-page  summary  of  each  individual  student's  strengths  and 
weaknesses  which  are  available  at  a glance  on  the  Objectives  Competency  Report 
(individual  student  report)  or  in  more  detail  on  the  Group  Response  Record  (item 
analysis).  If  a teacher  wishes  more  detailed  information  and  sample  remedial 
exercises  reference  to  the  Class  Management  Guide. 

When  assessing  the  utility  of  the  DEGEM  diagnostic  results,  it  is  important  to 
consider  the  accuracy  of  the  initial  level  of  placement  for  the  student.  It  is  important 
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that  the  student  is  placed  at  a level  where  he  is  able  to  succeed,  but  where  he  is 
being  "stretched"  or  challenged.  The  three  case  studies  seem  to  indicate  that 
DEGEM  may  be  placing  the  students  at  a level  where  they  are  not  challenged. 
There  is  some  evidence  that  DEGEM  may  be  under  assessing  the  student's  current 
level  of  achievement.  This  was  apparent  in  that  DEGEM  often  showed  the  student 
was  working  at  a specific  level,  yet  the  CAT  results  indicated  that  the  student  was 
able  to  correctly  answer  questions  at,  and  above,  the  level  specified  by  DEGEM.  In 
addition,  topics  which  may  be  closed  to  a student  because  of  his  overall  grade  level 
score,  were  areas  where  the  student  demonstrated  some  level  of  competency  on  the 
CAT.  One  possible  explanation  may  be  that  DEGEM  presents  a number  of  the 
same  types  of  questions  to  students  and  unless  the  students  can  demonstrate 
mastery  they  are  dropped  a level.  The  CAT  presents  one  or  two  questions  for  the 
student  in  the  area,  and  perhaps  having  fewer  questions  on  a topic  works  in  the 
students  favor.  Another  possibility  is  that  the  DEGEM  system  requires  the  student 
to  solve  the  questions  mentally,  while  the  CAT  allows  the  use  of  pencil  and  paper. 
Perhaps  DEGEM  is  measuring  memory  skills  as  well  as  mathematical  or  language 
skills.  This  may  explain  how  a student  can  succeed  in  one  area  (CAT)  and  fail  in 
another  area  (DEGEM)  when  the  areas  are  at  the  same  level  of  difficulty. 

A third  explanation  arises;  it  is  possible  that  security  at  sign-on  is  a problem  with 
the  system.  If  so,  this  may  adversely  effect  the  accuracy  of  the  system’s  results. 
During  several  observations  of  computer  use,  it  was  noticed  that  a number  of 
students  signed-on  with  an  I.D.  number  belonging  to  their  friends  or  relatives. 
Alternatively,  sign-on  could  be  achieved  by  randomly  entering  numbers  (four  or 
five  digit)  until  a student  was  able  to  sign-on.  The  actual  frequency  of  this 
occurrence  is  unknown;  however,  at  the  individual  level  this  would  greatly  affect 
the  diagnostic  results  for  a student.  If  this  were  a common  occurrence,  there  would 
be  implications  at  the  class  level. 

Cases  where  students  were  randomly  keying  responses  until  they  got  the  correct 
answer  were  also  observed.  This  seemed  to  occur  with  the  younger  students, 
especially  with  word  problems.  If  the  child  could  not  read  the  word  problem,  yet 
sees  the  numbers  "4"  and  "3"  he  would  enter  "7".  If  the  computer  indicated  error, 
he  would  enter  "1".  This  would  be  seen  from  the  computer's  diagnostic 
perspective,  as  an  inability  to  subtract  single  digit  numbers  less  than  five,  when  in 
fact  it  is  a reading  comprehension  problem.  Randomly  keying  responses  would 
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also  impact  the  validity  of  the  diagnostic  system’s  results;  patterns  which  emerge 
would  not  be  patterns  at  all.  This  could  also  account  for  possible  error  in  the 
diagnostic  capability  of  the  system. 

It  was  also  observed  that  there  were  some  students  who  did  not  seem  to  take  their 
time  on  the  computer  too  seriously,  that  is,  they  were  inattentive,  punching  keys 
randomly,  and  generally  off-task,  regardless  of  the  questions.  This  observation 
was  supported  in  part  by  the  time  on  task  data,  which  indicated  that  there  was  no 
significant  difference  in  time  on  task  on  the  computer  versus  the  class,  and  by  the 
teacher  questionnaire  data  which  indicated  that  teachers  perceived  that  eleven  to 
twenty  percent  of  their  students  were  off-task  while  working  with  DEGEM.  These 
behaviours  could  invalidate  the  diagnostic  results  provided  by  the  system. 
Therefore,  students'  perceptions  of  the  system's  function  and  student  motivation 
are  important  factors  which  can  affect  the  quality  of  input  into  the  system.  This 
directly  affects  the  quality  of  the  system's  output  (i.e.,  diagnosis).  This  is  also  true 
for  other  diagnostic  instruments;  however,  the  testing  environment,  combined  with 
closer  teacher  supervision  and  scrutiny,  tends  to  minimize  some  of  the  effects 
discussed  above.  All,  or  any  combination  of  the  above,  could  be  a possible 
explanation  for  lower  performance  results  on  the  DEGEM  system. 

Another  important  issue  to  consider  is  the  comprehensiveness  of  the  diagnostic 
results  provided  by  DEGEM.  It  is  preferred  that  this  system  provide  a 
comprehensive  overview  of  each  student’s  level  of  achievement,  and  specific  areas 
requiring  attention  in  both  mathematics  and  language  arts.  As  discussed  earlier,  the 
DEGEM  system  heavily  emphasizes  the  skill  components  in  these  subjects  and 
places  less  emphasis  on  the  concepts.  Therefore,  it  provides  less  information 
regarding  the  student's  weaknesses  at  a conceptual  level.  This  requires  the  teacher 
to  use  other  diagnostic  assessment  instruments  to  obtain  this  information.  This  can 
be  problematic  and  time-consuming;  thus,  may  result  in  a heavier  emphasis  on 
skills,  by  "default".  Hence,  it  is  possible  that  remediation  may  be  "skewed"  in 
favor  of  skills,  with  an  insufficient  emphasis  on  concepts.  This  may  be  considered 
a significant  limitation  of  the  DEGEM  diagnostic  system.  Additionally,  it  is 
unfortunate  that  the  DEGEM  system  itself  provides  no  remedial  sub-routines  for 
students  to  follow.  If,  in  fact,  the  system  is  able  to  pinpoint  areas  of  difficulty,  it 
would  seem  within  the  scope  of  current  technology  to  incorporate  sub-routines 
which  would  also  provide  individual  remediation  for  the  student. 
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The  final  issue  to  consider  in  determining  the  adequacy  of  the  DEGEM  diagnostic 
system,  is  its  ease  of  use  for  the  classroom  teacher.  As  noted  in  the  results  section 
when  reporting  the  results  of  the  teacher  questionnaire,  many  teachers  found  the 
DEGEM  reports  difficult  to  interpret.  This  limits  teacher  use,  and  therefore  the 
effectiveness  of  the  diagnostics.  During  the  preparation  of  the  DEGEM  case  study 
profiles,  the  process  of  interpreting  the  printouts  was  extremely  time-consuming 
and  frustrating,  requiring  continual  reference  from  one  source  to  another  (i.e., 
printout,  teacher’s  manual,  and  exercise  summary  book).  To  prepare  a written 
diagnostic  profile  using  DEGEM  results  is  estimated  to  take  at  least  an  hour.  This 
time  would  decrease  with  familiarity  with  the  materials;  however,  the  number  of 
cross  references  required  would  still  utilize  a considerable  amount  of  time.  As  well, 
it  must  be  remembered,  that  when  the  diagnostic  profile  is  completed,  one  must 
then  develop  a remedial  program,  search  out,  and  acquire  suitable  materials  for 
remediation. 

One  serious  weakness  which  interferes  with  the  ease  of  use  of  the  printout,  is  that 
although  the  same  coded  symbols  are  used  for  both  the  mathematics  and  language 
arts  printouts,  they  do  not  necessarily  have  the  same  meanings,  either  within  a 
printout,  or  between  printouts.  For  example  the  ”>"  symbol  on  the  mathematics 
printout  has  two  meanings.  If  it  appears  to  the  right  of  the  TO/ER  column  it  means 
the  time  allotment  for  that  student  has  been  increased  or  decreased.  If  it  appears  to 
the  right  of  an  exercise  level,  it  means  that  the  student  is  "stuck"  and  requires 
assistance.  It  also  serves  as  a decimal  point  when  the  teacher  looks  the  question  up 
in  the  exercise  manual  (i.e.,  31>2  becomes  31.2  in  the  exercise  manual).  In  the 
language  arts  section  ">"  means  that  the  student  is  having  problems  in  the  third 
group  of  five  exercises  (other  symbols  are  used  to  denote  the  first  and  second 
group).  The  symbols  appear  to  have  been  arbitrarily  assigned,  making  it  difficult  to 
remember  them  (as  opposed  to  using  symbols  which  would  have  some  link  with 
what  they  indicate).  As  well,  the  issue  is  further  confused  by  giving  the  same 
symbol  different  meanings.  Since  the  classroom  teacher  is  faced  with  expanding 
roles  and  responsibilities  in  education,  it  is  imperative  that  a diagnostic  system  is 
understandable  and  efficient  to  use.  By  comparison,  the  CAT  diagnostic  system 
was  easier  to  understand,  and  less  time-consuming. 
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Another  important  feature  for  the  teacher,  when  employing  a diagnostic  system,  is 
the  system's  provision  for  remediation  strategies.  Once  teachers  know  what  the 
problem  is,  they  want  to  know  what  to  do  about  it.  Other  than  providing  drill  and 
practice  in  the  area  of  deficit,  the  DEGEM  system  does  not  provide  any  support  in 
this  area.  The  teacher  is  left  to  find  appropriate  strategies  and  materials  to  use  with 
the  student  in  remediation.  This  is  a very  time-consuming  process  which  requires 
suitable  and  available  resources.  In  view  of  the  other  time  pressures  on  the  teacher, 
this  may  or  may  not  happen.  As  reported  on  the  teacher  questionnaire,  most 
teachers  agreed  that  they  now  had  a better  understanding  of  their  students' 
weaknesses,  they  also  agreed  that  they  did  not  know  what  to  do  about  them,  or 
how  to  begin  remediation.  This  can  be  considered  a serious  shortcoming  of  the 
DEGEM  system's  diagnostic  capability.  In  contrast,  the  CAT  provides  specific 
sample  strategies  for  remediation,  for  three  grade  ranges,  in  the  Class  Management 
Guide. 

In  conclusion,  the  DEGEM  system  appears  to  have  some  appropriate  diagnostic 
capabilities,  especially  in  the  skill  areas  of  mathematics  (computation)  and  language 
arts  (language  mechanics  and  language  expression).  Its  diagnostic  capability  in 
conceptual  areas  is  limited.  It  appears  to  have  the  student  working  at  a lower  skiU 
level  than  the  CAT  generally  indicates.  The  system  focuses  on  weaknesses  rather 
than  strengths  and  provides  no  suggestions  for  remediation.  The  DEGEM  reports 
are  difficult  to  interpret  and  require  great  familiarity  with  the  mathematics  and 
English  programs  (i.e.,  fifteen  topics  in  each).  System  security,  student 
perceptions,  and  student  motivation  may  be  important  factors  affecting  the  validity 
of  the  system's  diagnostic  results. 
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Question  Four  - How  useful  and  appropriate  are  the  DEGEM  CAI  inservice 
training  preparations  perceived  to  be  by  the  teachers  at  Kehewin  and  Saddle  Lake 
relative  to  system  management,  system  use,  and  authoring  courseware? 

During  the  first  year  of  the  project  (1987-1988),  a significant  portion  of  the  teacher 
questionnaire  was  committed  to  the  issue  of  inservice.  The  interim  report  indicated 
that  the  effectiveness  of  the  DEGEM  system  was  dependent,  to  a large  extent,  upon 
the  individual  teacher's  understanding  of  how  to  fully  utilize  the  system.  It  was 
apparent  from  both  the  questionnaires  and  face-to-face  interviews  that  this  objective 
had  not  been  realized.  Therefore,  one  of  the  most  crucial  recommendations  made 
when  the  interim  report  was  submitted,  was  for  more  inservice  time  to  be  arranged 
for  teachers.  This  fact  was  also  highlighted  with  the  respective  administrations, 
teachers,  and  with  the  Steering  Committee.  When  the  final  round  of  data  were 
being  collected,  both  schools  indicated  that  no  teacher  inservices  had  been 
conducted  at  either  site  during  the  1988-1989  school  year.  At  Kehewin  a new 
systems  manager  who  began  in  September  1988,  had  been  given  virtually  no 
inservice.  A second  systems  manager  was  hired  in  February  1989.  He,  however, 
received  some  assistance  from  the  systems  manager  who  was  employed  during  the 
1987-1988  school  year.  As  no  teacher  inservice  had  been  conducted  since  the 
interim  report,  no  further  questions  were  included  on  the  teacher  questionnaire 
regarding  inservice;  therefore,  what  follows  are  the  results  of  the  1987-1988  data 
collection.  It  is  important  to  note  that  the  recommendations  for  continued  inservice 
are  still  very  much  appropriate. 

This  qualitative  data  was  gathered  at  both  sites  by  means  of  a structured 
questionnaire  late  in  the  1987-1988  school  year.  In  addition  to  the  teaching  staff, 
the  systems  managers  and  administrators  from  both  schools  were  invited  to 
participate.  All  but  one  administrator  responded  by  filling  out  the  questionnaire.  In 
addition,  a structured,  open-ended,  face-to-face  interview  was  conducted  with  all 
teachers.  It  must  be  remembered  that  the  results  for  this  section  reflect  the  views  of 
the  respective  staffs  in  1988.  This  may  or  may  not  be  representative  of  their  current 
views.  This  is  especially  true  at  Kehewin  where  there  has  been  a significant  staff 
turnover  since  June  1988.  At  the  suggestion  of  the  Steering  Committee,  the 
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questions  regarding  inservice  were  divided  into  two  groups;  the  inservice  given  by 
the  DEGEM  personnel,  and  the  inservice  given  by  the  local  systems  manager.  The 
questions  devoted  to  the  topic  of  inservice  appear  in  Table  6,  with  the  mean  values 
for  each  group  of  teachers  reported  (Kehewin  teachers'  results  are  reported  under 
the  column  heading  "Keh",  and  the  Onchaminahos  teachers'  results  are  reported  in 
the  column  titled  "One"). 

It  should  be  noted  that  the  staff  at  Kehewin  have  received  more  inservice  than  the 
staff  at  Onchaminahos.  This  is,  in  part,  due  to  the  fact  that  the  system  has  been  in 
place  one  year  longer  at  Kehewin.  There  was  little  or  no  training  conducted  prior  to 
the  arrival  of  the  systems  in  either  school,  as  the  systems  arrived  with  little  advance 
warning.  The  systems  managers  were  selected  in  a relatively  arbitrary  manner,  and 
the  staffs  and  administrations  did  not  appear  to  receive  any  training  in  advance  of 
the  arrival  of  the  systems. 

Generally,  teachers  tended  to  agree  more  with  the  global,  "motherhood"  type 
questions  (e.g.,  "I  was  satisfied  with  the  level  of  training  I received  regarding  the 
DEGEM  computer")  than  with  specific,  focused  questions,  (e.g.,  "As  a result  of 
the  training,  I have  an  adequate  knowledge  of  the  operation  of  the  student 
diagnostic  system").  Mean  values  for  each  of  the  questions,  by  school,  were 
obtained.  This  allows  for  comparison  of  teacher  perceptions  between  schools.  T- 
tests  were  run  on  the  mean  values  to  determine  whether  significant  differences 
(p<0.001)  occurred  in  the  teachers  responses.  Generally,  it  appears  that  the 
teachers  at  Kehewin  are  more  positively  predisposed  to  the  DEGEM  system  than 
those  at  Onchaminahos.  Regardless  of  any  differences,  both  staffs  were  clear  in 
their  desire  for  additional  and  regular  inservice. 

The  specific  discussion  of  inservice  will  begin  by  examining  the  inservice  delivered 
by  the  DEGEM  staff.  Both  groups  of  teachers  were  generally  satisfied  with  the 
level  of  training  they  received  and  indicated  that  their  comfort  level  with  DEGEM 
was  enhanced.  When  asked  whether  the  DEGEM  system  worked  as  well  as  they 
were  told  it  would,  there  was  a significant  difference  between  the  groups;  the 
Kehewin  teachers  agreed  while  the  Onchaminahos  teachers  tended  to  disagree.  The 
administrators  who  responded  were  reversed  with  Kehewin  responding  lower  and 
Onchaminahos  rating  it  higher.  The  same  split  was  evident  when  the  administrators 
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were  asked  whether  their  teachers  were  satisfied  with  the  level  of  training  they 
received. 

A series  of  questions  were  asked  of  teachers  as  to  whether  or  not  they  have 
increased  their  knowledge  of  specific  aspects  of  the  system  as  a result  of  the 
inservice.  Responses  with  respect  to  the  operation  of  the  hardware,  software, 
authoring  system,  student  diagnostic  system  and  registering  a class  ranged  from 
general  disagreement  to  neutrality.  The  sole  exception  being  that  teachers  stated 
they  had  a better  knowledge  of  the  recordkeeping  functions  as  a result  of  the 
inservice.  Administrators  indicated  that  they  did  not  have  an  adequate  knowledge 
of  the  recordkeeping  functions  as  a result  of  the  inservice.  When  asked  whether 
there  were  major  content  areas  that  were  not  covered  in  the  training,  the  teacher's 
responses  were  somewhat  ambivalent  with  the  Onchaminahos  teachers  indicating 
that  it  was  more  likely  sections  were  omitted.  There  was  general  satisfaction  with 
the  level  of  expertise  of  those  conducting  the  inservice.  With  respect  to  the  pacing 
and  information  presented,  Kehewin  teachers  tended  to  respond  slightly  more 
favorably.  One  administrator  observed  that,  "The  use  of  the  system  depends  a great 
deal  on  how  comfortable  the  teacher  is  using  a computer  . . . more  time  should  be 
spent  in  encouraging  staff  to  be  computer  literate." 

The  following  section  reports  teacher  responses  to  the  inservice  provided  locally  . 
Perhaps  the  clearest  statement  made  by  teachers  with  respect  to  inservice  was  made 
in  this  section.  When  asked  to  respond  to  "I  would  not  find  periodic  upgrading 
useful",  both  groups  strongly  disagreed.  The  same  questions  regarding  areas  such 
as  hardware  and  software  were  asked  in  this  section.  The  responses  were  very 
similar  to  the  section  above,  generally  somewhat  higher  (perhaps  0.1),  but 
reflecting  the  same  pattern.  Again,  both  groups  were  satisfied  with  the  level  of 
expertise  of  those  providing  the  inservice.  There  was  a significant  difference 
between  groups  when  asked,  "I  understand  the  intended  use  of  DEGEM  in  my 
school",  with  the  Kehewin  group  reporting  more  agreement.  The  administrators 
responded  to  this  question  positively,  with  the  administration  at  Onchaminahos 
indicating  stronger  agreement . 

When  the  systems  managers  were  asked  if  they  were  aware  how  the  system  was  to 
be  implemented  in  the  school,  their  responses  differed,  with  the  Kehewin  systems 
manager  indicating  more  agreement.  When  asked  if  they  had  all  the  information 


37 


they  required  prior  to  the  systems  arriving  in  the  school,  both  strongly  disagreed. 
They  were  also  asked  if  they  now  had  all  the  information  they  required  and  the 
response  was  general  agreement.  They  indicated  some  agreement  with  the 
statement  that  they  have  received  adequate  training  from  the  DEGEM  staff  and  both 
agreed  that  their  training  differed  significantly  from  the  teachers’  training.  The 
suggestion  was  made  by  one  systems  manager  that  during  future  inservices  teachers 
could  benefit  from  a greater  emphasis  on  how  system  use  could  be  better  integrated 
into  the  teaching  of  subject  matter  in  the  classroom. 

On  the  open-ended  portion  of  the  teacher  questionnaire,  the  comments  indicated  that 
teachers  wanted  more  time  for  inservice  and  a greater  depth  of  understanding  the 
system.  One  teacher  indicated,  "Basically  we  were  given  a student  level  experience 
on  the  computer",  another  stated,  "I  did  receive  some  training  - basically  how  to 
turn  the  thing  on.  I have  no  idea  how  to  obtain  different  questions  etc."  Several 
commented  that  the  inservice  was  "quick  and  superficial"  and  more  time  is  required. 
As  well,  follow-up  sessions  were  requested;  "Short  sessions  on  a regular  basis 
would  allow  us  to  apply  what  we  are  learning".  One  teacher  stated  that  the  "hands- 
on  experience  was  good,  but  we  did  not  learn  anything  about  individualizing  or 
setting  programs".  There  did  not  appear  to  be  any  specific  differences  in  the  written 
comments  between  the  two  schools. 


Table  6 

Mean  Values  of  Inservice  Questions  - Teacher  Questionnaire 


1988 

Keh  One 


31.  I was  satisfied  with  the  level  of  training  I received  regarding 
the  DEGEM  computer  system. 


2.3  2.3 


32.  I found  my  comfort  level  with  the  DEGEM  system  was 
enhanced  by  the  training  I received. 


2.6  2.6 


33.  I saw  new  ways  to  deliver  the  curriculum  to  my  students 
using  the  DEGEM  system. 


1.9  1.4 


34.  The  DEGEM  system  has  worked  as  well  they  told  me  it 
would. 


3.0  1.5 
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Table  6 Continued 

Mean  Values  of  Inservice  Questions  - Teacher  Questionnaire 


35.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of 
the  operation  of  the  DEGEM  HARDWARE. 

36.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of 
the  operation  of  the  DEGEM  SOFTWARE. 

37.  The  training  has  provided  me  with  adequate  knowledge  of 
the  operation  of  the  DEGEM  record  keeping  functions. 

38.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of 
the  operation  of  the  DEGEM  authoring  system. 

39.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of 
the  operation  of  the  DEGEM  student  diagnostic  system. 

40.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of 
how  to  register  my  class  on  the  DEGEM  system. 

4 1 . Since  I was  training  on  the  DEGEM  system  I have  found 
new  uses  for  the  system  that  were  not  covered  in  the 
training. 

42.  There  were  major  content  areas  not  covered  in  the  training. 

43.  I suffered  from  information  overload  during  the  training. 

44.  I found  the  pacing  of  the  training  to  be  inappropriate. 

45.  I was  satisfied  with  the  expertise  of  the  individuals  who  did 
the  training. 

46.  I would  not  find  periodic  upgrading  useful. 

47.  I found  my  comfort  level  with  the  DEGEM  system  was 
enhanced  by  the  training  I received. 

48.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of 
the  operation  of  the  DEGEM  HARDWARE. 

49.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of 
the  operation  of  the  DEGEM  SOFTWARE. 


1988 

Keh  One 

1.1  2.4 

1.8  1.9 

2.3  2.1 

1.0  1.5 
1.8  1.6 

1.1  1.6 

1.9  .9 
-2.0  2.5 

2.0  1.9 

2.4  1.7 

2.9  3.0 
0.4  0.6 

2.7  2.4 
1.3  2.0 

1.7  1.6 
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Table  6 Continued 

Mean  Values  of  Inservice  Questions  - Teacher  Questionnaire 

1988 


Keh 

One 

50. 

The  training  has  provided  me  with  adequate  knowledge  of 
the  operation  of  the  DEGEM  record  keeping  functions. 

2.3 

2.0 

51. 

As  a result  of  the  training,  I have  an  adequate  knowledge  of 
the  operation  of  the  DEGEM  authoring  system. 

1.2 

1.3 

52. 

As  a result  of  the  training,  I have  an  adequate  knowledge  of 
the  operation  of  the  DEGEM  student  diagnostic  system. 

2.8 

1.6 

53. 

As  a result  of  the  training,  I have  an  adequate  knowledge  of 
how  to  register  my  class  on  the  DEGEM  system. 

1.3 

1.1 

54. 

I suffered  from  information  overload  during  the  training. 

1.3 

1.9 

55. 

I found  the  pacing  of  the  training  to  be  inappropriate. 

1.4 

1.9 

56. 

I was  satisfied  with  the  expertise  of  the  individuals  who  did 
the  training. 

3.3 

2.9 

57. 

I understand  the  intended  use  of  the  DEGEM  system  in  my 
school. 

3.1 

2.0 

DISCUSSION:  Question  FOUR 

In  1988  the  teachers  in  both  schools  demonstrated  a positive  response  toward  the 
inservice  provided.  There  were  no  significant  differences  portrayed  between  the 
quality  of  inservice  provided  by  the  DEGEM  system  people,  or  locally,  by  the 
systems  managers  at  each  school.  Significant  differences  between  the  teacher 
groups  was  noted  with  the  Kehewin  teachers  generally  being  more  positively 
oriented  toward  the  system.  There  are  several  hypotheses  which  could  be  tendered 
to  account  for  the  difference.  First,  the  Kehewin  teachers  have  received  more 
inservice  with  respect  to  DEGEM;  therefore,  perhaps  they  are  more  comfortable 
with  the  system  and  understand  it  better.  Second,  the  Kehewin  teachers  have  used 
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the  system  for  just  over  one  year  longer  than  the  Onchaminahos  teachers.  This 
exposure  could  account  for  the  difference  in  orientation.  Third,  and  most  likely, 
there  was  a major  system  failure  at  Onchaminahos  last  spring  (1988).  Eventually, 
all  the  student  data  were  lost,  and  the  system  did  not  function  well  for  several 
weeks.  Students  were  "rolled  back"  in  the  topics  they  were  working  on,  causing 
much  frustration  for  all  concerned.  Finally,  all  the  data  were  entered  manually  for 
each  student  by  the  systems  manager,  and  the  system  was  again  operational,  toward 
the  end  of  the  school  year.  Unfortunately,  as  no  inservice  was  conducted  in  the 
subsequent  year,  it  is  difficult  to  select  a "most  likely"  hypothesis. 

When  teachers  were  asked  about  specific  areas  where  they  had  acquired  knowledge 
about  DEGEM,  they  appeared  to  have  some  difficulty  pinpointing  where  their 
knowledge  base  had  been  enhanced.  However,  when  asked  general  questions  the 
responses  were  quite  positive.  From  this  it  would  appear  that  generally,  teachers 
want  to  have  the  system  in  the  school,  but  are  not  familiar  enough  with  the  system 
to  utilize  it  in  any  sort  of  diagnostic  sense.  This  was  confirmed  during  the 
structured  interviews,  although  several  teachers  indicated  they  used  the  system  to 
diagnose  student  needs,  very  few  were  able  to  give  any  type  of  specific  indication 
of  how  that  was  being  done. 

With  respect  to  system  use,  it  appears  that  all  teachers  are  able  to  use  the  system. 
That  is,  they  can  turn  it  on  and  have  their  students  follow  the  program.  With 
respect  to  system  management,  that  is  altering  the  system  for  their  own  specific 
needs,  or  readily  understanding  and  implementing  the  diagnostic  printout,  there  is  a 
varied  but  generally  poor  ability  to  do  so.  Authoring  is  a feature  not  used  by 
anyone,  systems  managers  nor  teachers.  When  responding  to  some  of  the  same 
questions  asked  of  teachers,  the  administrators  did  not  always  seem  to  have  an 
accurate  perception  of  how  their  staff  used,  or  perceived,  the  system.  This  could  be 
in  part  due  to  the  fact  that  only  one  administrator  actually  used  the  system  and  had 
first  hand  knowledge. 
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Question  Five  - Does  the  use  of  DEGEM  CAI  result  in  improvements  in 
students'  thinking  skills  (cognitive  strategies  skills)  relative  to  conventional 
instruction? 

The  Test  of  Cognitive  Skills  (TCS)  was  selected  as  the  instrument  used  to  measure 
thinking  skills.  It  was  administered  to  all  students  who  wrote  the  CAT  with  the 
exception  of  some  of  the  special  education  students  whose  teachers  chose  not  to 
administer  it.  The  TCS  was  administered  during  the  same  time  frame  as  the  CAT. 
Most  students  who  wrote  the  CAT  wrote  the  TCS  although  this  is  not  universally 
true.  As  with  the  CAT,  no  clear  trends  have  emerged  with  respect  to  the  effect  of 
CAI  on  thinking  skills. 

In  1988,  the  results  comparing  Grade  2A  with  Grade  2B,  Grade  3 A with  Grade 
3B,  Grade  4 with  Grade  5,  and  Grade  5 with  Grade  6,  are  reported  in  Table  7.  The 
Grade  7 class  was  not  included  in  this  study.  The  TCS  results  reported  are  based 
on  the  average  raw  scores  for  each  class  on  five  measures  - sequencing,  analogies, 
memory,  verbal  reasoning,  and  total  battery. 

The  Grade  2A  (n=18)  class  is  stronger  on  all  measures  than  the  Grade  2B  (n=18) 
class  except  for  memory,  where  the  2B  class  scored  higher.  There  were  no 
significant  differences  between  the  groups  except  on  the  sequencing  subtest  (in 
favor  of  2A). 

The  Grade  3 A (n=22)  class  is  stronger  on  all  measures  than  the  Grade  3B  (n=16) 
class  except  for  sequencing,  where  the  3B  class  score  is  statistically  significantly 
better  than  the  3A  class.  On  the  analogies  subtest,  the  3A  class  demonstrates  a 
statistically  significandy  higher  score. 

The  Grade  4 (n=22)  class  shows  no  statistically  significant  differences  when 
compared  to  the  Grade  5 class;  however,  they  demonstrate  higher  means  on  the 
sequencing  and  verbal  reasoning  subtests.  The  Grade  5 (n=19)  class  shows  better 
scores  on  analogies,  memory,  and  the  total  battery. 
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Table? 

Mean  Scores  on  the  Test  of  Cognitive  Skills  Reported  Bv  Grade 


Grade  2A  By  Grade  2B 


Area 

Means 

Standard  Dev. 

T 

Prob.2- 

2A 

2B 

2A 

2B 

Sequencing 

441.5 

372.3 

81.9 

94.5 

2.3 

0.03 

Analogies 

406.5 

378.9 

147.4 

99.2 

0.7 

0.52 

Memory 

582.7 

634.6 

99.0 

70.3 

-1.8 

0.08 

Verbal  Reasoning 

435.4 

411.7 

115.2 

122.7 

0.6 

0.56 

TOTAL 

468.6 

449.5 

95.8 

70.6 

0.7 

0.51 

Grade  3A  By  Grade  3B 

Area 

Means 

Standard  Dev. 

T 

Prob.2- 

3A 

3B 

3A 

3B 

Sequencing 

361.6 

443.7 

119.1 

70.4 

2.3 

0.03 

Analogies 

436.4 

332.0 

85.6 

203.9 

2.1 

0.05 

Memory 

513.5 

496.5 

59.3 

43.4 

0.9 

0.36 

Verbal  Reasoning 

458.2 

412.0 

83.1 

123.4 

1.3 

0.19 

TOTAL 

444.4 

434.5 

52.8 

74.6 

0.5 

0.66 

Grade  4 By  Grade  5 

Area 

Means 

Standard  Dev. 

T 

Prob.2- 

4 

5 

4 

5 

Sequencing 

448.1 

420.6 

106.7 

110.1 

0.7 

0.50 

Analogies 

379.5 

416.1 

120.3 

77.0 

-1.0 

0.35 

Memory 

497.9 

522.5 

73.5 

79.0 

-0.9 

0.38 

Verbal  Reasoning 

476.9 

465.5 

500.4 

92.2 

0.4 

0.66 

TOTAL 

451.1 

457.5 

56.2 

77.0 

-0.3 

0.80 
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Table  7 Continued 

Mean  Scores  on  the  Test  of  Cognitive  Skills  Reported  By  Grade 


Grade  5 By  Grade  6 
Area 


Means  Standard  Dev.  T Prob.2-Tail 


5 6 5 6 


Sequencing 

Analogies 

Memory 

Verbal  Reasoning 


420.6  570.7  110.1  103.6  -3.8  0.00 

416.1  532.2  77.0  106.6  -3.3  0.00 

522.5  704.2  79.0  99.7  -5.4  0.00 

465.5  520.8  92.2  69.6  -1.9  0.07 

457.5  587.1  77.0  67.6  -4.8  0.00 


TOTAL 


The  Grade  3 A (n=22)  class  is  stronger  on  all  measures  than  the  Grade  3B  (n=16) 
class  except  for  sequencing,  where  the  3B  class  score  is  statistically  significantly 
better  than  the  3A  class.  On  the  analogies  subtest,  the  3A  class  demonstrates  a 
statistically  significantly  higher  score. 

The  Grade  4 (n=22)  class  shows  no  statistically  significant  differences  when 
compared  to  the  Grade  5 class;  however,  they  demonstrate  higher  means  on  the 
sequencing  and  verbal  reasoning  subtests.  The  Grade  5 (n=19)  class  shows  better 
scores  on  analogies,  memory,  and  the  total  battery. 

The  Grade  5 (n=19)  class  is  weaker  than  the  Grade  6 (n=19)  class  on  all  measures. 
In  fact,  the  Grade  6 group  scored  statistically  significantly  better  on  all  measures 
except  verbal  reasoning. 

Correlations  were  developed  between  the  total  battery  score  on  the  CAT  and  the 
four  subtest  scores  on  the  TCS  as  well  as  the  total  score  on  the  TCS.  It  is  expected 
that  the  correlations  would  be  in  the  0.7  range,  as  that  is  the  correlation  figure 
generally  accepted  when  discussing  the  relationship  between  intelligence  and 
achievement.  Table  8,  presents  the  correlation  figures  between  the  total  battery  raw 
score  on  the  CAT  and  the  total  raw.score  on  the  TCS. 
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Table  8 

Correlations  of  Total  Scores  Bv  Gmde_Qn  the  Canadian  Achievement  Test  And  The 


Test  of  Co^itive  Skills 


Onchaminahos  School 

Correlation 

Kehewin  School 

Correlation 

Grade  2 

0.57 

Grade  2 

0.40 

Grade  3 

0.54 

Grade  3 

0.25 

Grade  4 

0.56 

Grade  4 

0.90 

Grade  5 

0.89 

Grade  5 

0.85 

Grade  6 

0.72 

Grade  6 

0.82 

TOTAL  Grades  2-6 

0.56 

TOTAL  Grades  2-6 

0.54 

Caution  must  be  exercised  in  interpretation,  as  the  sizes  of  each  class,  hence  the 
total  number  of  subjects  in  each  school,  are  relatively  small.  It  can  be  seen  however 
that  the  correlations  appear  to  be  much  stronger  in  the  higher  grades,  and  that  the 
total  correlation  for  all  grades  combined  is  somewhat  lower  than  would  be 
expected. 

Qualitative  information  was  gathered  from  teachers  in  each  school  with  respect  to 
thinking  skills  and  is  reported  in  Table  9.  When  asked  if  there  had  been  an 
improvement  in  their  students'  thinking  skills  (1988),  Kehewin  teachers  tended  to 
agree  while  Onchaminahos  teachers  tended  to  disagree.  In  1989  the  Onchaminahos 
teachers'  responses  were  somewhat  closer  to  the  1988  Kehewin  teachers' 
responses.  With  respect  to  the  way  that  their  students  approach  problems  since 
being  on  DEGEM,  both  groups  saw  their  strongest  students  having  derived  the 
greatest  benefit,  followed  by  their  average  students,  and  finally,  their  weaker 
students  benefiting  least.  In  1988  there  were  no  large  differences  between  the 
scores.  In  1989  the  Onchaminahos  teachers  indicated  much  stronger  agreement  that 
DEGEM  assisted  both  the  stronger  and  the  average  students,  while  they  continued 
to  disagree  that  the  system  was  an  asset  to  the  weaker  group.  Both  groups  tended 
to  agree  that  their  students  were  better  able  to  sequence  information  since  using  the 
system. 
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Table  9 

Mean  Values  of  Thinking  Skills  Questions  - Teacher  Questionnaire 

1988  1989 

Keh  One  One 


58. 


59. 


60. 


61. 


62. 


63. 


64. 


Sinee  my  students  have  been  on  the  DEGEM 
system,  I have  notieed  an  improvement  in  their 
thinking  skills. 

Sinee  my  students  have  been  on  the  DEGEM 
system,  I have  not  notieed  any  ehanges  in  the  way 
they  approaeh  problems  they  eneounter. 

Sinee  my  students  have  been  on  the  DEGEM 
system,  I have  notieed  that  my  stronger  students 
approaeh  problems  they  eneounter  in  a superior 
way. 

Sinee  my  students  have  been  on  the  DEGEM 
system,  I have  notieed  that  my  average  students 
approaeh  problems  they  eneounter  in  a superior 
way. 

Sinee  my  students  have  been  on  the  DEGEM 
system,  I have  notieed  that  my  weaker  students 
approaeh  problems  they  eneounter  in  a superior 


2.4  1.8  2.1 


1.6  2.3  1.6 


2.0  2.1  2.9 


1.9  1.9  2.9 


way. 

1.6 

1.6 

1.6 

My  students  are  better  able  to  sequenee  sinee  using 
the  DEGEM  system. 

2.4 

2.0 

2.6 

My  students  who  are  better  able  to  sequenee  benefit 
more  from  the  DEGEM  program. 

2.1 

2.5 

2.5 

Five  teaehers  made  additional  written  eomments  (1988)  with  respeet  to  this  issue. 
One  person  indieated  that  no  differenee  has  been  notieed.  Another  indieated  a slight 
improvement  in  problem  solving  ability.  The  third  suggested  that  the  ehildren  are 
more  eonfident  on  the  system  and,  therefore,  attempt  more  problems.  Two  saw 
ehanges  but  eould  not  attribute  them  to  DEGEM;  "I  eonfess  to  not  really  notieing 
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improvement  which  could  be  attributable  to  the  use  of  the  system",  and  "It  is 
difficult  to  state  changes  are  solely  the  result  of  the  system ...  we  have  put  in  place 
new  mathematics,  language  arts  and  science  programs." 


DISCUSSION:  Question  RvE 

This  question  is  similar  to  Question  One,  except  thinking  skills  rather  than  student 
achievement  are  the  issue.  One  thing  that  is  clear,  is  that,  thinking  skills  are  harder 
for  teachers  to  measure,  even  informally,  than  is  achievement.  The  qualitative  data 
were  gathered  from  both  sites  while  quantitative  data  were  received  only  from 
Onchaminahos  School. 

With  respect  to  the  qualitative  data,  there  was  modest  agreement  that  DEGEM  was 
having  a positive  effect  on  thinking  skills.  Teachers  seemed  to  have  difficulty  with 
this  section,  as  only  five  of  the  seventeen  wrote  any  comments  at  all  (1988),  and 
only  one  indicated  that  any  changes  in  thinking  skills  could  be  attributed  to 
DEGEM.  Over  time  the  Onchaminahos  teachers  tended  to  respond  similarly, 
although  in  the  area  of  assisting  stronger  and  average  students,  their  responses  were 
more  positive. 

The  quantitative  data  that  were  available  are  insufficient  to  yield  any  differences  or 
trends.  The  correlations  run  between  the  CAT  and  the  TCS  show  a large  degree  of 
variability  among  grade  levels.  Unfortunately,  as  no  useable  quantitative  data  from 
1989  are  available  due  to  cohort  contamination  and  subjects  leaving  the  study,  no 
conclusions  regarding  the  effect  of  DEGEM  on  thinking  skills  can  be  drawn.  Even 
the  qualitative  information  does  not  yield  any  definitive  statement  as  to  whether 
these  students’  thinking  skills  were  enhanced  by  their  exposure  to  DEGEM.  The 
one  area  where  a trend  emerges  is  that  the  teachers  agree  that  the  stronger  and 
average  students  derive  the  most  benefit  from  working  on  the  system,  while  the 
weaker  students  benefit  the  least 
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Question  Six  - To  what  extent  are  teachers  and  administrators  able  to  implement 
the  DEGEM  CAI  program  as  intended  with  respect  to  activities  such  as  the 
following:  individualized  instruction;  diagnosing  students'  strengths  and 
weaknesses  and  assigning  appropriate  entry  level  tasks;  supplementing, 
complementing,  and  integrating  CAI  with  conventional  and  creative  teaching; 
reteaching  as  required;  evaluating  and  recording  pupil  progress;  and  applying 
mastery  learning  principles?  What  are  teachers'  perceptions  about  the 
appropriateness  and  usefulness  of  the  courseware?  What  are  teachers'  perceptions 
about  changes  in  their  relationships  with  pupils  and  approaches  to  teaching  which 
are  attributable  to  the  project? 

The  meaning  of  "intent"  (i.e.,  intended  by  whom)  is  the  focal  point  of  this  question. 
It  appears  that  there  are  two  levels  of  intent.  The  first  level  is  that  of  those  who 
developed  the  system.  According  to  Dr.  R.  Stock  (1988),  DEGEM  is  intended  to 
provide  students  with  drill  and  practice  in  the  areas  of  language  arts  and 
mathematics.  It  is  also  capable  of  other  functions  should  a school  be  interested  in 
other  applications  (e.g.,  typing,  word  processing,  vocational  English).  She 
indicated  that  DEGEM  is  not  intended  to  replace  microcomputers  currently  found  in 
schools  (e.g..  Apples,  IBM's).  Rather  it  occupies  a separate  niche  in  the  school. 
The  DEGEM  computer  is  not  used  to  "teach"  as  such,  but  for  reinforcement  and 
drill. 

In  May  1988,  each  school  was  also  asked  to  submit  a statement  of  intent  with 
respect  to  DEGEM.  Neither  school  had  such  a document  at  the  time  of  request; 
however,  both  systems  managers  were  able  to  draft  a statement.  No  updates  or 
revisions  were  made  to  these  statements  in  1989.  Kehewin  identified  five 
objectives  for  the  DEGEM  system  in  the  school: 

1 . To  operate  as  a diagnostic  tool  in  assessing  the  strengths  and  weaknesses  of 
students  in  mathematics,  English  and  reading. 

2.  To  provide  extensive  reinforcement  of  teacher-taught  lessons  in  mathematics 
and  English. 

3 . To  maintain  specific  skills  in  mathematics,  English,  and  reading. 
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4 . To  provide  a tool  for  programming  and  problem  solving. 

5 . To  develop  skills  in  typing  and  word  processing. 

Onchaminahos  School  provided  objectives  under  three  headings: 

Qgner^  ObjecMs: 

1 .  To  experiment  with  an  advanced  CAI  system. 

Long  Term  Obiectives: 

1 . To  utilize  the  system  in  the  teaching  of  computing  and  programming  skills. 

2.  To  implement  word  processing  for  use  in  English  language  arts  classes  at  the 
junior  and  senior  high  level. 

3 . To  develop  programs  to  assist  in  the  teaching  of  Cree  language. 

Immediate  Objectives: 

1 . To  enhance  teachers'  effectiveness  in  the  teaching  of  basic  math  and  English 
skills  to  elementary  and  junior  high  students. 

2.  To  objectively  determine  individual  student  levels  of  skills  and  monitor  student 
progress. 

3 . To  implement  computer-based  typing  and  job  skills  programs. 

Diagnosis  and  complementing  teacher  instruction  are  two  common  functions 
identified  by  each  school. 

Physically,  the  systems  are  set  up  differently  in  each  school.  In  Onchaminahos, 
each  student  terminal  is  set  in  a study  carrel  while  at  Kehewin  the  terminals  are 
sitting  on  long  tables  with  alternate  terminals  facing  one  side  of  the  table  or  the 
other.  In  addition,  at  Kehewin  there  is  a Commodore  64  in  the  room.  In  1988,  this 
micro  was  available  to  be  used  to  teach  specific  concepts  to  the  class  before  they 
used  DEGEM  for  skill  reinforcement.  In  both  schools  the  student  sessions  on 
DEGEM  are  set  to  10  minutes.  In  1988,  Kehewin  blocked  their  classes  in  for  15 
minutes  while  Onchaminahos  uses  20-minute  blocks.  At  Kehewin,  students  are 
restricted  from  accessing  any  topics  which  are  not  being  taught  in  the  classroom 
while  at  Onchaminahos  the  students  are  free  to  work  on  any  topics  the  computer 
allows  (by  their  assigned  grade  level).  In  1989,  as  Kehewin  had  two  systems 
managers,  neither  of  whom  was  initially  familiar  with  the  systems,  it  appears  that 
student  use  was  sporadic  until  February  1989. 
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In  summary,  the  system  is  intended  to  provide  drill  and  practice  activities  to 
students  on  topics  in  which  they  have  received  previous  instruction.  It  is  not 
intended  to  provide  instruction.  A second  feature  of  the  computer  systems  is  that  it 
is  able  to  diagnose  student  problems.  The  system  does  not  provide  remediation. 
That  is  the  function  of  the  teacher.  These  two  intended  uses  seem  to  be  consistent 
with  the  information  provided  by  Dr.  Stock  and  the  two  systems  managers. 

The  information  used  to  answer  question  six  is  qualitative  and  was  gathered  from 
both  sites.  The  primary  method  was  the  teacher  questionnaire  which  had  seventy 
items  that  were  related  to  this  question.  Again,  the  general  trend  of  teachers 
agreeing  more  with  general  ’’motherhood"  statements  and  less  with  specific 
focussed  statements  is  seen. 

In  1988,  the  teachers,  administrators,  and  systems  managers  were  first  given  a 
series  of  general  questions  regarding  DEGEM.  In  1989,  the  only  results  reported 
are  for  Onchaminahos  teachers  for  the  reasons  stated  earlier.  All  results  for  this 
section  are  reported  in  Table  10.  In  1988  eight  teachers  returned  questionnaires 
from  Kehewin  and  nine  teachers  returned  questionnaires  from  Onchaminahos;  two 
administrators  from  Onchaminahos  and  one  from  Kehewin  responded  as  did  the 
system  managers  for  each  school.  In  1989  a total  of  two  individuals  (teachers, 
systems  manager,  and  administrators)  returned  questionnaires  from  Kehewin,  and 
only  one  was  completed.  Therefore  no  useable  results  can  be  reported  from 
Kehewin.  The  combined  return  from  Onchaminahos  was  nine  individuals.  No 
written  comments  were  submitted  from  either  school. 

In  1988,  all  groups  agreed  that  the  DEGEM  system  was  an  educational  asset  and 
that  it  made  their  job  at  school  easier.  The  Onchaminahos  teachers,  in  1989, 
continued  to  support  the  concept  that  the  system  is  an  educational  asset;  however, 
their  responses  were  in  the  neutral  range  with  respect  to  making  their  job  easier.  In 
1988  there  was  some  support  {neutral  to  agree)  for  the  concept  of  wanting  to  see  the 
system  used  in  other  core  subjects;  and  less  support  for  using  the  system  with 
complementary  subjects.  In  the  following  year  Onchaminahos  teachers’  responses 
ranged  from  neutral  to  disagree  with  respect  to  core  subjects  and  from  disagree  to 
strongly  disagree  in  the  area  of  complementary  subjects.  When  asked  if  the 
computer  system  speeds  up  the  learning  process  (1988)  the  responses  were  mixed 
with  the  teachers  generally  agreeing  with  the  statement,  the  systems  managers 
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strongly  agreeing  and  the  administrators  generally  disagreeing.  The  more  recent 
results  from  Onchaminahos  suggest  that  those  teachers  now  tend  to  disagree  with 
the  statement.  There  was  a significant  difference  between  the  two  groups  of 
teachers  when  asked  if  the  system  frustrates  them,  with  the  Kehewin  teachers 
appearing  less  frustrated.  The  administrators  indicated  they  were  the  most 
frustrated  (neutral  to  agree),  while  the  systems  managers  disagreed  with  the 
statement.  The  Onchaminahos  group  recently  indicated  slightly  more  frustration 
with  the  system.  When  asked  whether  the  DEGEM  computer  did  everything  they 
thought  it  would,  the  groups  responded  in  the  neutral  to  disagree  range  with  the 
Kehewin  teachers  being  most  positive.  The  1989  results  show  the  Onchaminahos 
teachers  are  still  in  the  disagree  to  neutral  range.  When  asked  if  there  should  be 
more  DEGEM  computers  in  the  school,  it  appears  that  the  systems  managers  and 
administrators  favored  the  concept  more  than  the  teachers,  and  more  recently  the 
Onchaminahos  teachers  expressed  somewhat  more  interest  in  the  concept.  There 
was  general  disagreement  among  the  groups  when  asked  if  the  system  works  the 
way  it  is  supposed  to  all  the  time;  however  the  two  groups  of  teachers  differed 
significantly,  with  the  Kehewin  group  being  the  only  one  that  agreed  with  this 
statement.  In  1989  the  results  suggest  that  the  teachers  are  more  critical  of  the 
system  in  this  area.  There  was  also  a significant  difference  between  the  teacher 
groups  as  to  whether  DEGEM  wastes  time,  although  the  general  consensus  among 
all  groups  was  that  it  does  not.  There  has  been  little  change  in  this  area  when  the 
1989  results  are  examined. 

With  respect  to  the  teachers'  impressions  regarding  the  capability  of  the  DEGEM 
system  in  diagnosing  students'  needs,  when  asked  about  the  system's  ability  to 
diagnose  problems,  there  was  general  agreement  in  1988;  however,  in  1989  the 
teachers  responded  in  the  disagree  to  neutral  range.  As  to  which  student  group  is 
most  effectively  diagnosed,  Kehewin  teachers  ranked  them  in  order  as:  weaker 
students,  stronger  students,  and  average  students;  with  responses  ranging  from 
disagree  to  agree.  In  1988  the  Onchaminahos  teachers  ranked  them  in  order  as: 
stronger,  weaker,  and  average.  In  1989  they  saw  the  ranking  as:  average,  stronger, 
and  weaker,  with  large  changes  in  ratings  for  each  group.  In  both  cases  (1988),  the 
average  students  were  seen  as  deriving  the  least  benefit;  whereas  in  1989  this  group 
was  seen  as  receiving  the  most  benefit.  The  Kehewin  teachers  tended  to  indicate 
stronger  agreement  to  the  suggestion  that  the  DEGEM  information  agreed  with 
other  student  data  they  have.  The  Onchaminahos  teachers'  responses  have  not 
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changed  over  time.  In  1988  both  groups  agreed  that  the  DEGEM  information  was 
consistent  with  their  perceptions  regarding  student  needs;  however  in  1989  the 
indication  was  that  this  was  not  the  case.  In  1988  they  also  stated  that  DEGEM 
diagnosis  assisted  in  attending  to  the  educational  needs  of  the  students;  again 
however,  there  was  a sharp  drop  in  support  for  that  concept  in  1989.  The  response 
to,  "I  understand  how  to  use  the  DEGEM  printout  to  diagnose  my  students’  needs” 
was  agree.in  1988;  however  it  had  dropped  to  neutral  in  1989.  The  administrators 
at  both  schools  agreed  that  the  information  from  DEGEM  was  consistent  with  other 
information  they  had  on  the  students,  and  that  DEGEM  diagnosis  assists  teachers  in 
attending  to  the  educational  needs  of  their  students.  One  administrator  indicated  that 
the  greatest  weakness  regarding  diagnosing  student  achievement  is  that  the  system 
does  not  remediate  the  deficit  it  identifies;  therefore,  the  teacher  is  left  to  develop  an 
individual  plan. 

DEGEM  was  also  seen  as  enabling  the  teacher  to  provide  individualized  instruction 
for  students  in  1988,  while  in  1989  the  teachers'  responses  were  much  less 
favorable.  When  asked  if  their  students  receive  less  individualized  instruction  since 
DEGEM's  arrival  (1988),  both  groups  indicated  they  disagreed.  In  1989  the 
response  was  unchanged.  On  the  question,  "I  individualize  instruction  for  each  of 
my  students  in  the  regular  classroom  based  upon  the  DEGEM  results”,  the 
responses  were  in  the  neutral  to  disagree  range  for  both  years.  The  administrators 
(1988),  however,  indicated  that  they  thought  the  system  had  enhanced  the  teachers' 
individualized  instructional  planning.  The  Kehewin  teachers  saw  the  system's 
ability  to  diagnose  student  strengths  and  weaknesses  as  approximately  equal. 
When  answering  the  same  questions  Onchaminahos  teachers  saw  the  system  as 
being  less  able  to  diagnose  strengths  than  weaknesses;  this  was  the  case  in  both 
years.  When  asked  if  the  system  could  remediate  student  deficits  there  was  general 
disagreement.  There  was  general  agreement  that  DEGEM  provided  information  to 
assist  the  teacher  in  remediating  the  deficits.  Again,  there  was  little  difference 
across  time. 

A series  of  questions  were  asked  regarding  assigning  student  entry  levels.  The 
1988  responses  indicated  that  generally,  teachers  saw  themselves  as  unable  to 
assign  entry  levels,  yet  when  the  computer  did,  there  was  ambivalence  regarding 
the  appropriateness.  In  1989  there  was  a major  change  in  that  the  Onchaminahos 
teachers  indicated  not  only  were  they  capable  of  assigning  student  entry  levels,  they 
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preferred  to  do  so  and  had  more  confidence  in  their  judgement  in  this  area  than  in 
the  computers'  judgement. 

Questions  were  asked  which  reflected  potential  changes  in  teaching  style  since 
DEGEM  arrived.  Teachers  were  asked  if  they  had  changed  the  way  they  teach;  they 
responded  in  the  negative  (both  groups  and  over  time).  When  asked  if  the  DEGEM 
system  was  incompatible  with  their  teaching  style,  there  was  general  disagreement 
at  Kehewin  while  the  Onchaminahos  group  tended  to  be  and  still  is  neutral  in  their 
response.  When  asked  if  they  had  less  reteaching  to  do,  both  groups  disagreed  in 
1988,  and  this  was  supported  in  1989.  There  was  a significant  difference  in 
responses  (1988)  to  the  question,  "I  use  the  DEGEM  system  information  to 
determine  which  concepts  require  reteaching  in  the  classroom"  with  Kehewin 
teachers  tending  to  agree  while  the  other  group  tended  toward  neutrality.  This  trend 
continued  in  1989.  Three  questions  were  asked  about  DEGEM's  ability  to  assist 
the  teacher  in  knowing  when,  how,  and  what,  to  reteach.  Both  groups  agreed  that 
the  system  does  not  assist  them  in  knowing  how  to  re  teach  material.  This  was  true 
in  1989  as  well.  With  respect  to  the  "when"  and  "what"  the  Kehewin  teachers' 
responses  tended  toward  agree  while  the  Onchaminahos  teachers'  responses  tended 
toward  neutral.  In  1989  the  pattern  for  the  Onchaminahos  group  was  similar  to  the 
previous  year. 

The  next  set  of  questions  on  the  teacher  questionnaire  dealt  with  the  system's  ability 
to  assist  in  evaluating  student  progress.  Both  groups  tended  to  agree  that  the 
system  assists  in  evaluating  student  progress;  however,  in  1989  the  Onchaminahos 
group  was  much  less  supportive  of  this  concept.  When  asked  to  identify  in  which 
specific  areas  it  was  helpful,  the  teachers  appeared  to  have  some  difficulty.  With 
respect  to  assigning  grades,  retention/promotion,  and  assigning  report  card  marks, 
there  was  general  agreement  that  the  use  of  DEGEM  results  should  be  limited  (both 
groups  and  across  time).  In  1988,  with  respect  to  retention/promotion,  the 
administrators  were  asked  to  respond  to  the  following,  "The  DEGEM  system 
results  should  not  be  used  to  assist  in  determining  student  retention/promotion.". 
Their  mean  response  was  neutral;  however  there  was  a large  difference  in  the 
response  from  the  administrator  who  also  taught  using  DEGEM  strongly  agree  and 
the  others  disagree.  When  asked  if  report  card  marks  should  reflect  DEGEM 
results,  administrators  tended  to  disagree. 
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When  teachers  were  asked  if  they  used  the  DEGEM  grade  equivalent  scores  in 
determining  how  students  were  achieving,  the  Kehewin  teachers  tended  to  agree, 
while  the  Onchaminahos  teachers  tended  to  be  more  neutral.  Teachers  at  Kehewin 
were  less  likely  to  use  the  changes  in  grade  equivalent  scores  while  Onchaminahos 
teachers  were  slightly  more  likely  to  use  the  change  in  scores. 

The  next  set  of  questions  focused  on  the  appropriateness  of  courseware  and  the 
teachers  relationships  and  interactions  with  the  students.  With  respect  to  the 
appropriateness  of  the  courseware,  three  areas  were  explored;  the  developmental 
level,  the  cultural  needs,  and  the  grade  level.  Generally,  the  Kehewin  teachers  saw 
the  courseware  as  being  more  appropriate.  Both  groups  indicated  that  the  strongest 
area  is  the  developmental  component.  This  trend  continued  into  1989.  The 
weakest  area  was  seen  as  the  cultural  appropriateness  of  the  courseware,  although 
the  Onchaminahos  teachers'  ratings  improved  in  this  area  in  1989.  Grade  level 
ranked  between  the  cultural  aspects  and  the  developmental  level.  There  was  a 
significant  difference  (1988)  in  teacher  perception  regarding  the  appropriateness  of 
DEGEM  with  respect  to  the  cultural  needs  of  the  students,  with  the  Onchaminahos 
group  assigning  a lower  value.  With  respect  to  student  relationships,  the  teachers 
in  both  schools  saw  no  differences  in  their  relationships  to  various  groups  of 
students  (stronger,  average,  weaker).  There  was  a trend  among  the  Onchaminahos 
teachers  which  continued  into  1989  in  that  they  saw  their  relationships  with  the 
stronger  students  as  benefitting  most.  With  respect  to  interacting  more  or  less  with 
students  there  was  general  disagreement  that  using  DEGEM  had  any  effect. 

To  the  statement,  "I  present  the  topic  in  the  classroom  before  I start  my  students  on 
the  DEGEM  system",  the  teachers  generally  disagreed.  As  well,  there  has  been  no 
change  over  time.  Teachers  indicated  (1988)  that  between  ten  and  twenty  percent  of 
the  students  had  difficulty  paying  attention  during  the  10-minute  session.  It  appears 
that  this  may  be  slightly  higher  in  1989. 

Another  series  of  questions  asked  the  teachers  how  they  saw  their  students  reacting 
to  various  features  of  the  DEGEM  system.  When  using  the  system  for  arithmetic, 
answers  must  be  entered  right  to  left.  Teachers  were  asked  if  the  students  found 
that  confusing.  The  Kehewin  teachers  responded  neutrally  while  the 
Onchaminahos  teachers  tended  to  agree.  In  1989,  their  response  had  moved  to 
strongly  agree.  Both  groups  were  asked  if  they  thought  DEGEM  asked  questions 
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differently  than  the  texts  being  used.  Again  the  same  trend  was  seen,  including  the 
fact  that  the  Onchaminahos  scores  have  increased.  On  the  issue  of  whether  this  was 
confusing  to  students,  the  trend  held.  However  the  Onchaminahos  teachers  did  not 
see  this  as  large  a problem  as  last  year. 

There  were  questions  asked  that  were  unique  to  the  administrators.  When  asked  if 
DEGEM  had  created  timetabling  difficulties,  all  disagreed.  They  also  disagreed  that 
the  system  has  caused  an  increase  in  supplementary  resource  materials.  They 
agreed  that  the  teachers  perceive  their  computer  time  allotment  to  be  adequate. 
There  was  some  disagreement  as  to  whether  the  advent  of  DEGEM  has  resulted  in  a 
change  in  teaching  styles  (Mean  2.0;  Range  1 - 3)  or  whether  the  printouts  assist  in 
monitoring  how  teachers  are  meeting  student  needs  (Mean  2.0;  Range  1 -3). 


Table  10 

Mean  Values  of  General.  Diagnostic  and  Teacher  Application  Questions  - Teacher 
Questionnaire 


1988 

1989 

Keh 

One 

One 

1. 

The  DEGEM  system  is  an  educational  asset. 

3.5 

3.3 

2.9 

2. 

The  DEGEM  system  makes  my  job  at  school  easier. 

2.5 

2.4 

2.0 

3. 

I would  like  to  see  the  DEGEM  system  used  in  other 

core  subjects  like  social  studies  and  science. 

2.1 

2.6 

1.7 

4. 

I would  like  to  see  the  DEGEM  system  used  in 
complementary  subject  areas  like  art  and  music. 

2.1 

1.9 

0.6 

5. 

The  DEGEM  system  speeds  up  the  learning  process. 

2.8 

2.6 

1.6 

6. 

The  DEGEM  system  frustrates  me. 

0.6 

1.9 

1.8 

7. 

The  DEGEM  system  does  everything  I thought  it 
would. 

2.4 

1.5 

1.8 

8. 

The  DEGEM  system  is  great! 

3.0 

2.7 

1.6 

9. 

We  should  have  more  DEGEM  computer  terminals  in 
our  school. 

2.0 

1.9 

2.3 

10. 

The  DEGEM  computer  works  the  way  it  is  supposed 
to  all  the  time. 

2.5 

1.2 

0.9 
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Table  10  Continued 

Mgan  Yalyes  of  General,  Diagnostic  and  Tgagbgr  Appligatipn  .Qyg?tiQns  - Tgacbgr 
Questionnaire 


1988  1989 

Keh  One  One 


11. 

I think  the  DEGEM  system  wastes  time. 

0.4 

1.6 

1.4 

21. 

I think  that  the  DEGEM  system  is  able  to  adequately 
diagnose  the  problems  my  students  have. 

2.5 

3.0 

1.4 

22. 

The  DEGEM  diagnosis  is  adequate  for  my  weaker 
students. 

29. 

2.1 

1.1 

23. 

The  DEGEM  diagnosis  is  not  adequate  for  my  average 
students. 

1.1 

1.6 

2.4 

24. 

The  DEGEM  diagnosis  is  adequate  for  my  stronger 
students. 

2.5 

3.6 

1.4 

25. 

The  DEGEM  diagnosis  supports  (is  consistent  with) 
other  information  I have  on  my  students. 

3.1 

2.6 

2.6 

26. 

The  DEGEM  diagnosis  is  not  consistent  with  my 
impressions  of  my  students. 

0.9 

1.6 

2.8 

27. 

The  DEGEM  diagnosis  helps  me  in  attending  to  the 
educational  needs  of  my  students. 

2.8 

3.1 

1.4 

28. 

I understand  how  to  use  the  DEGEM  printout  to 
diagnose  my  students’  needs. 

3.1 

3.1 

2.0 

29. 

Information  that  DEGEM  gives  me  on  my  students  is 
not  compatible  with  the  information  I get  from  the 
CTBS. 

1.1 

1.0 

2.3 

30. 

The  DEGEM  system  ranks  students  in  the  same  order 
I do. 

2.5 

1.4 

2.9 

65. 

The  DEGEM  system  allows  me  to  provide 
individualized  instruction  for  my  students. 

3.0 

2.6 

1.5 

66. 

I individualize  instruction  for  more  of  my  students 
since  using  the  DEGEM  system. 

2.3 

1.8 

1.5 
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Table  10  Continued 

Mean  Values  of  General.  Diagnostic  and  Teacher  Application  Questions  - Teacher 
Questionnaire 


1988  1989 


Keh 

One 

One 

67. 

In  my  classroom,  students  receive  less  individualized 
instruction  now  than  they  did  prior  to  the  arrival  of  the 
DEGEM  system.  1 . 1 

1.1 

1.0 

68. 

I individualize  instruction  for  each  of  my  students  in 
the  regular  classroom  based  upon  the  DEGEM  system 
results.  1.9 

1.5 

1.8 

69. 

The  DEGEM  system  does  not  assist  me  in  diagnosing 
my  students’ strengths.  1.3 

2.1 

2.1 

70. 

The  DEGEM  system  does  not  assist  me  in  diagnosing 
my  students'  weaknesses.  1.5 

1.4 

1.9 

71. 

The  DEGEM  system  provides  me  with  information 
that  allows  me  to  remediate  my  students'  deficits.  2.8 

2.1 

2.5 

72. 

The  DEGEM  system  is  able  to  remediate  my  students' 
deficits.  1.8 

1.3 

1.1 

73. 

I am  capable  of  assigning  my  student’s  entry  level 
tasks  on  the  DEGEM  system.  1 .4 

" 1.9 

3.1 

74. 

When  I assign  the  student  entry  level  it  is  appropriate. 

2.0 

2.0 

3.8 

75. 

When  the  DEGEM  system  assigns  the  student  entry 
level  it  is  appropriate.  2.4 

2.6 

2.8 

76. 

I prefer  to  assign  the  student  entry  level  myself  rather 
than  let  the  computer  do  it.  1.8 

1.6 

2.6 

77. 

Since  the  DEGEM  system  has  arrived  I have  changed 
the  way  I teach  in  my  classroom.  1 .5 

1.0 

0.8 

78. 

The  DEGEM  system  is  not  compatible  with  my 
teaching  style.  1.3 

2.1 

2.0 

79. 

I am  using  my  students'  DEGEM  experience  to 
enhance  learning  in  the  classroom.  2.4 

2.6 

2.5 
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Table  10  Continued 

Mean  Values  of  General.  Diagnostic  and  Teacher  Application  Questions  - Teacher 
Questionnaire 


1988 

1989 

Keh 

One 

One 

80. 

Since  using  the  DEGEM  system,  I have  more 
flexibility  in  how  I can  teach.  1 . 8 

2.0 

1.6 

81. 

Since  my  students  have  been  on  the  DEGEM  system, 

I find  I have  less  re- teaching  to  do.  1.8 

1.7 

1.3 

82. 

I use  the  DEGEM  system  information  to  determine 
which  concepts  require  re-teaching  in  the  classroom.  2.9 

1.9 

1.9 

83. 

Since  the  DEGEM  system  has  arrived  I am  better  able 
to  know  when  to  re-teach  a concept.  2 . 5 

2.1 

1.9 

84. 

Since  the  DEGEM  system  has  arrived  I am  better  able 
to  know  how  to  re-teach  a concept.  1 .5 

1.4 

0.6 

85. 

Since  the  DEGEM  system  has  arrived  I am  better  able 
to  know  what  material  to  re-teach.  2.9 

2.3 

2.0 

86. 

The  DEGEM  system  assists  me  in  evaluating  student 
progress.  3.0 

2.4 

1.6 

87. 

The  information  the  DEGEM  system  provides  assists 
me  when  I am  assigning  student  grades.  1.8 

1.6 

1.5 

88. 

The  DEGEM  diagnostic  student  results  should  be  used 
to  assist  in  determining  student  retention/promotion. 

1.6 

1.0 

0.8 

89. 

The  DEGEM  system  assists  me  in  recording  student 
progress.  2.4 

2.0 

1.8 

90. 

Report  card  marks  should  not  reflect  DEGEM  system 
results.  2.4 

3.1 

3.0 

91. 

I change  the  DEGEM  system  parameters  to  change  the 
aspects  of  the  drill.  2.4 

1.2 

2.5 

92. 

The  DEGEM  courseware  is  inappropriate  for  the 
curriculum  at  the  grade  level  I am  teaching.  1 .5 

1.4 

2.0 
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Table  10  Continued 

Mean  Values  of  General.  Diagnostic  and  Teacher  Application  Questions  - Teacher 
Questionnaire 


1988  1989 


Keh 

One 

One 

93. 

The  DEGEM  courseware  is  appropriate  for  the 
developmental  level  of  the  students  in  the  grade  level  I 
am  teaching.  3.0 

2.1 

2.4 

94. 

The  DEGEM  courseware  is  appropriate  to  the  cultural 
needs  of  the  students  at  the  grade  level  I am  teaching. 

2.3 

0.7 

1.8 

95. 

The  DEGEM  system  will  enhance  learning  for  all 
students  in  the  school.  2.6 

2.0 

1.6 

96. 

Since  using  the  DEGEM  system,  I find  that  my 
relationship  with  my  stronger  students  has  changed 
for  the  better.  1.5 

2.0 

2.0 

97. 

Since  using  the  DEGEM  system,  I find  that  my 
relationship  with  my  weaker  students  has  changed  for 
the  better.  1.5 

1.4 

1.9 

98. 

Since  using  the  DEGEM  system,  I find  that  my 
relationship  with  my  average  students  has  changed  for 
the  better.  1.5 

1.3 

1.8 

99. 

Since  the  DEGEM  system  has  been  in  place  I interact 
more  with  each  of  my  students.  2.0 

1.6 

1.0 

100. 

Since  the  DEGEM  system  has  been  in  place  I interact 
less  with  each  of  my  students.  1 .4 

1.1 

0.6 

101. 

I present  the  topic  in  the  classroom  before  I start  my 
students  on  the  DEGEM  system.  1 .8 

1.4 

1.4 

102. 

Since  using  the  DEGEM  system  I find  that  I do  not 
have  to  do  as  much  classroom  teaching  on  how  to 
apply  the  concept.  1 . 8 

1.6 

0.8 
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Table  10  Continued 

Mean  Values  of  General.  Diagnostic  and  Teacher  Application  Questions 
Questionnaire 


1988 


Keh 


103.  In  arithmetic,  the  fact  that  some  of  the  answers  are 
entered  "left  to  right"  and  some  are  entered  "right  to 

left"  confuses  my  students.  2.1 

1 04.  The  students  in  my  classroom  frequently  exceed  the 
limits  of  the  DEGEM  system,  that  is  go  off  the  high 

end  of  the  scale.  1 .0 

1 05 . My  weaker  students  are  academically  too  weak  to 

benefit  from  their  time  on  the  DEGEM  computer.  1 . 3 

1 06.  My  average  students  are  academically  too  weak  to 
benefit  from  their  time  on  the  DEGEM  computer.  0.9 

1 07 . My  stronger  students  are  academically  too  weak  to 
benefit  from  their  time  on  the  DEGEM  computer.  0 . 6 

108.  percent  of  my  students  have  trouble  paying 

attention  (are  off  task)  during  the  10  minute  session.  1 .8 


109.  I use  the  DEGEM  system  grade  equivalent  scores  in 

determining  how  my  students  are  achieving.  2.8 

110.  I use  the  change  in  grade  equivalent  scores  reported  by 
the  DEGEM  system  to  determine  student  achievement. 

2.0 

111.  The  way  DEGEM  system  asks  questions  regarding 
subject  matter  is  different  than  the  way  the  textbooks 

ask  questions.  1 .9 

112.  The  way  the  DEGEM  system  asks  questions  confuses 

many  of  my  students.  1.8 


One 


3.0 

1.2 

2.1 
1.6 
1.1 
2.2 
2.0 

2.3 

2.9 

2.4 


1989 

One 

4.0 

0.6 

1.8 

1.5 

1.3 

2.5 
1.8 

2.3 

3.1 


2.4 
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The  following  section  is  a summary  of  the  open-ended  questions  to  which  the 
teachers  responded  on  their  questionnaires.  However,  it  should  be  noted  that  the 
results  presented  in  this  section  are  from  1988.  The  reason  for  this  is  that  no  actual 
teacher  questionnaires  were  returned  in  1989,  only  the  answer  sheets;  therefore,  no 
written  comments  were  available.  With  respect  to  diagnosis,  DEGEM's  greatest 
strength  is  seen  as  "variety";  "It  reinforces  my  perceptions";  "rank  according  to 
needs";  "gives  a detailed  breakdown  of  skills";  "pinpoints  difficult  areas";  "fosters 
speed  and  accuracy";  "doesn't  take  personalities  into  account".  The  area  which 
caused  teachers  the  most  difficulty  was  "understanding  the  computer  printout  - how 
ironic";  "some  English  areas  have  too  few  questions";  "students  doing  harder  work 
need  help  more  often  than  students  at  the  lower  levels";  "diagnose  by  stanine  level, 
ranked  in  Alberta";  "remembering  what  the  symbols  mean  - include  a legend  in 
every  printout.";  "grade  level  equivalences  are  incorrect,  especially  in  division  one". 

Teachers  were  asked  to  respond  to  several  stems: 

a)  If  I could  change  one  thing  about  how  the  system  is  being  used  in  this  school  I 
would  . . .:  "make  grade  equivalences  closer  to  Alberta  norms";  "have  more 
inservice  on  using  results,  especially  over  time";  "isolate  each  student";  "like  to 
have  access  to  programs  in  other  subjects  (culture,  social)  and  have  programs 
adapted  to  particular  needs";  "coordinate  it  more  closely  with  the  textbooks  we 
use". 

b)  The  thing  I do  not  think  should  be  changed  . . .:  "the  availability  of  the 
experience";  "the  ten-minute  time  frame";  "the  system  as  a whole".  The  most 
useful  information  I get  from  the  system  is  . . .:  "urgency  of  need";  "grade- 
equivalent  scores";  "errors  over  questions  answered";  "diagnostic  ability"; 
"confirmation  of  mastery";  "early  detections  of  problems". 

c)  The  most  useful  information  I get  from  the  system  is  "urgency  of  need"; 
"grade-equivalent  scores";  "errors  over  questions  answered";  "diagnostic 
ability";  "confirmation  of  mastery";  "early  detections  of  problems". 

Several  general  comments  were  made:  "I  use  the  system  mainly  for  the  drill  aspect." 
"I  am  very  pleased  with  the  system.  It  does  more  than  I had  anticipated.",  "A  word 
processing  program  and  other  useful  software  would  help  maximize  student 
interaction.",  "A  better  student  reward  system."  "The  system  offers  little  feedback 
to  students." 
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Following  are  some  relevant  comments  from  administrators  and  systems  managers. 
One  administrator  said,  "If  I were  to  sum  up  the  impact  of  DEGEM  in  the  school,  I 
would  say  that  without  the  systems  manager  this  system  would  not  be  functioning 
as  effectively  as  it  is  today."  One  systems  manager  indicated  that  his  major  concern 
is  the  reliability  of  the  terminal  and  terminal  keypads.  As  well,  he  suggested  that  if 
a modem  could  be  used  with  the  system,  it  would  assist  in  the  area  of  software 
support. 

During  the  structured  interview  with  teachers,  some  of  the  same  topics  discussed 
above  were  covered  again;  however,  the  questions  allowed  for  more  input  from  the 
teachers.  Fifteen  interviews  were  conducted  in  both  schools  in  1988  and  seventeen 
in  1989  (nine  at  Kehewin  and  eight  at  Onchaminahos).  Four  teachers  were  using 
the  system  for  subjects  (usually  typing)  other  than  language  arts  or  mathematics  in 
1988  while  three  teachers  (all  at  Kehewin)  made  the  same  statement  in  1989.  Ten 
teachers  (1988)  indicated  DEGEM  has  impacted  their  teaching  style.  Nine  teachers 
in  1989  indicated  there  was  an  impact  on  their  teaching  style.  Comments  indicated 
that  it  allowed  teachers  to  focus  on  children's  problem  areas.  One  teacher  indicated 
that  subsequently  less  drill  was  done  in  the  classroom. 

When  teachers  were  asked  how  they  use  DEGEM  to  assist  in  their  teaching,  in 
1988  they  responded:  drill  (9);  diagnosis  (5);  review/reinforcement  (4);  remediation 
(3);  and  evaluation  (1).  In  1989  they  responded  drill  (14);  remediation  (10); 
diagnosis  (4);and  reward  (5).  Comments  ranged  from  positive,  "I  check  the 
weekly  print  out,  and  it  usually  corresponds  with  my  observation."  to  negative,  "I 
don't  trust  it;  is  the  readout  accurate  - no."  Most  comments  were  neutral  to 
positive. 

When  asked  how  easy  the  system  was  to  use  (1  low  and  10  high),  the  range  was 
from  5 to  10  with  a mean  of  8.2  in  1988  and  8.4  in  1989.  In  1988,  one  systems 
manager  indicated  that  as  a teacher,  in  terms  of  ease,  he  rated  the  system  8; 
however,  as  a systems  manager  he  rated  it  4. 

In  1988,  twelve  teachers  indicated  that  their  students  liked  DEGEM,  two  said  they 
did  not,  and  one  teacher  indicated  that  the  students  used  to  like  it  but  do  not  at  this 
point.  Twelve  teachers  indicated  that  certain  students  liked  the  system  more.  They 
indicated  that  it  was  the  top  students  or  the  more  studious  individuals  who  enjoyed 
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it.  Generally,  the  low  functioning  or  inattentive  students  were  perceived  as 
disliking  the  system.  In  1989,  thirteen  teachers  indicated  that  certain  students  like 
the  system  more.  Again,  the  comments  indicated  that  the  more  able  students 
seemed  to  enjoy  the  system  better;  however,  one  teacher  indicated  that  the  really 
bright  children  can  become  disinterested  due  to  the  relatively  long  wait  between 
questions. 

When  asked  whether  their  students  benefited  from  their  experience  with  the  system 
(1  low  and  10  high)  the  range  was  from  3 to  9 with  a mean  of  6.1  (1988).  All  but 
two  teachers  indicated  that  there  are  certain  groups  of  students  who  benefit  more. 
Of  the  thirteen  who  saw  specific  groups  benefit  more,  ten  stated  the  top  students 
gained  the  most,  two  stated  both  the  high  and  low  ends  gained  the  most,  and  one 
stated  the  high  and  average  gained  most.  The  range  in  1989  was  from  3 to  8 with  a 
mean  of  5.9  suggesting  a slight  downward  trend.  Again,  the  majority  of  the 
teachers  indicated  that  there  were  certain  groups  of  students  who  benefitted  more. 
Their  responses  indicated  that  the  better  students  gained  the  most.  One  teacher 
stated  that  the  "good  readers"  derived  the  most  benefit. 

Teacher  perception  with  respect  to  how  the  DEGEM  system  compares  to  other 
computer  systems  they  may  have  used  was  explored.  Teachers  were  first  asked  to 
rank  their  familiarity  with  computers  and  then  to  compare  DEGEM  to  other 
computers  they  have  used  (1  low;  10  high).  In  1988  the  responses  to  the  first 

question  ranged  from  1 to  10  with  a mean  of  4.9.  Eight  of  fifteen  teachers  rated 

( — 

themselves  3 or  less.  In  1989  the  range  was  from  1 to  9 with  a mean  of  5.1; 
however  only  four  teachers  rated  themselves  3 or  less.  The  responses  to  the  second 
question  (1988)  ranged  from  2 to  10  with  a mean  of  6.0  in  1989  and  from  1 to  10 
with  a mean  of  5.3.  It  would  appear  that  generally  the  teachers  are  seeing 
themselves  as  more  familiar  with  computers,  and  that  DEGEM  is  comparing  less 
favorably  to  other  systems  over  time.  It  should  be  noted  that  in  1988  only  eight 
teachers  felt  able  to  make  any  type  of  comparison  while  in  1989  fifteen  teachers 
were  able  to  do  so.  The  pattern  of  the  responses  for  those  teachers  is  interesting. 
In  Table  11  the  first  and  third  column  indicate  the  rank  the  teacher  gave  to  the 
question  regarding  familiarity  with  computers  (Familiar)  while  the  second  and 
fourth  columns  indicated  their  responses  to  how  DEGEM  compares  with  other 
systems  (Compare).  Please  note,  that  "teacher  1"  in  1988  is  not  the  same  individual 
as  "teacher  1"  in  1989. 
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It  can  be  seen  that  there  is  a somewhat  inverse  relationship  between  their  knowledge 
of  computers  and  viewing  DEGEM  as  comparing  favorably.  This  trend  was 
evident  in  both  years;  however,  the  sample  size  is  too  small  to  place  much  meaning 
in  this  tendency.  In  1988  the  comments  generally  indicated  that  the  most  positive 
feature  was  seen  as  ease  of  use  while  the  most  negative  features  included  lack  of 
graphics  and  lack  of  motivating  features  (games).  In  1989  those  two  issues  were 
also  raised.  In  addition  a benefit  mentioned  was  that  students  can  work  at  their  own 
speed  and  at  their  own  level,  while  a drawback  that  was  mentioned  was  the  lack  of 
teacher  knowledge  (grading  system,  report  printout,  programming). 


Table  11 

Relationship  Between  Familiarity  With  Computers  and  The  Perceived  Relative 
Value  of  DEGEM 


1988 

1989 

Teacher 

Familiar 

Compare 

Teacher 

Familiar 

Compare 

1 

3 

9 

1 

1 

8 

2 

3 

8 

2 

2 

5 

3 

3 

9 

3 

2 

5 

4 

5 

2 

4 

3 

1 

5 

7 

10 

5 

4 

5 

6 

7 

2 

6 

4 

6 

7 

8 

6 

7 

5 

9 

8 

10 

2 

8 

5 

10 

9 

5 

5 

10 

6 

3 

11 

6 

8 

12 

8 

8 

13 

8 

3 

14 

9 

2 

15 

9 

1 

64 


Teachers  were  asked  to  rate  the  impact  of  DEGEM  on  their  students'  learning.  In 
1988  the  scores  ranged  from  3 to  10  with  a mean  of  5.6.  Eight  of  the  fourteen 
teachers  who  answered  this  question  rated  DEGEM  5 or  less;  six  rated  it  6 or  more. 
In  1989  the  range  was  from  2 to  10  with  a mean  of  6.2  and  only  seven  of  the 
seventeen  teachers  rated  it  5 or  less.  In  1988  the  final  comments  appeared  equally 
split.  Two  teachers  indicated  that,  "It  is  a waste  of  money  - get  rid  of  it."  and  two 
teachers  indicated  that,  "It's  a fantastic  system;  I don't  want  to  be  without  it."  The 
rest  of  the  comments  were  in  between,  generally  reflecting  the  thinking  that  the 
system  has  more  potential  than  is  being  utilized.  In  1989  of  the  thirteen  teachers 
who  made  comments,  only  four  made  any  positive  comments.  They  tended  to  be 
quite  general,  (e.g.,  helps  in  reading,  seems  to  meet  needs,  its  a great  machine,  I 
like  it  for  what  I use  it  for,  depends  on  the  class).  The  negative  comments  were 
sometimes  more  specific  and  intense  (e.g.,  I do  not  like  it,  useless  in  reinforcing, 
uses  outdated  technology  to  operate  outdated  software  that  is  founded  on  outdated 
pedagogy).  Three  individuals  indicated  that  there  was  a need  for  inservice. 


DISCUSSION:  Question  Six 

The  qualitative  data  required  to  answer  this  question  were  gathered  from  both  sites. 
In  1988  it  was  determined  that  the  intent  of  the  system  was  to  provide  drill  and 
practice  activities  to  students  on  topics  in  which  they  have  received  previous 
instruction.  As  well,  the  system  is  intended  to  diagnose,  not  to  provide  remedial 
instruction. 

The  stakeholders  generally  indicated  that  they  liked  the  system  and  wanted  to 
continue  to  use  it.  As  well,  teachers  generally  agreed  that  the  system  was  able  to 
diagnose  students'  areas  of  need,  and  that  it  provided  them  with  information  which 
allowed  for  individualized  instruction.  However,  most  indicated  that  they  did  not 
use  the  information  collected  in  any  specific  way,  including  its  application  to 
individualized  instruction.  Entry  level  tasks  were  not  assigned  by  teachers.  In  all 
cases  the  computer  assigned  the  entry  level.  Further  it  appears  that  in  1989  that  the 
popularity  of  the  system  at  Onchaminahos  is  waning.  As  well  there  appeared  at 
both  schools  an  increased  desire  on  the  part  of  the  teachers  to  receive  additional 
inservice,  which  was  not  provided. 
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Most  teachers  indicated  that  they  have  not  changed  the  way  they  teach  since  the 
arrival  of  DEGEM  and  they  do  more  reteaching  now.  DEGEM  appears  to  assist 
teachers  in  knowing  "what"  and  "when"  to  reteach;  however,  it  does  not  seem  to 
help  in  the  area  of  "how"  to  reteach.  Both  groups  agreed  that  DEGEM  results  assist 
in  evaluating  students’  progress;  however,  there  were  no  clear  specific  areas  where 
there  was  agreement  in  how  the  results  should  be  applied.  It  was  clear  however, 
that  teachers  preferred  not  to  see  the  results  used  in  determining  report  card  marks 
or  assist  in  deciding  retention/promotion.  It  is  interesting  to  note  that  this 
perception  was  not  generally  shared  by  administration. 

The  courseware  seems  more  appropriate  for  the  developmental  level  and  the  grade 
level  of  the  students;  however,  teachers  did  not  see  it  as  culturally  appropriate, 
especially  at  Onchaminahos.  These  views  remained  stable  at  Onchaminahos  in 
1989.  With  respect  to  the  issue  of  changing  relationships  with  students,  teachers 
generally  saw  no  change,  except  perhaps  at  Onchaminahos  where  teachers  indicated 
that  they  had  a more  positive  relationship  with  the  stronger  students.  Neither  group 
saw  the  system  having  any  effect  on  the  amount  of  time  they  spent  interacting  with 
students. 

One  interesting  trend  emerged  during  the  structured  interviews.  There  seems  to  be 
an  inverse  relationship  between  knowledge  of  computers  and  positive  orientation 
toward  DEGEM.  That  is,  the  more  individuals  knew  about  computers,  the  less 
they  liked  DEGEM.  This  seemed  to  be  consistent  in  both  1988  and  1989,  although 
it  was  perhaps  stronger  in  the  first  year.  It  would  therefore  appear  that  the  more 
computer  literate  a teacher  is  (i.e.,  exposure  and  use  of  other  systems),  the  less 
likely  the  teacher  is  to  indicate  that  the  DEGEM  system  compares  favorably.  If 
however,  the  teacher  has  little  or  no  experience  with  computers,  the  DEGEM 
system  is  perceived  as  an  excellent  system. 
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Question  Seven  - To  what  extent  are  students  assigned  to  the  DEGEM  CAI 
Program  able  to  use  the  courseware  as  intended?  Do  they  appear  to  benefit  from  the 
feedback  provided?  Is  time  on  task  adequate?  How  long  does  it  take  for  students 
to  become  proficient  on  the  system?  What  do  pupils  perceive  to  be  the  relative 
merits  of  this  version  of  CAI  and  conventional  instruction?  How  do  pupils  perceive 
that  relationships  with  teachers  have  altered  as  a consequence  of  the  CAI  program 
experience? 

The  methodology  used  to  answer  this  question  was  similar  to  question  six. 
Questionnaires  were  administered  to  students  in  both  schools;  grades  3 to  7 at 
Onchaminahos  and  grades  3 to  8 at  Kehewin.  These  questionnaires  had  a Likert- 
type  scale  using  the  same  rating  as  for  the  teachers.  The  data  will  be  reported  in  the 
same  fashion  with  the  Kehewin  students'  means  first,  and  the  Onchaminahos 
students'  means  second.  As  well,  there  was  an  opportunity  for  students  to  respond 
to  some  open-ended  questions.  In  1988,  eighty  nine  students  responded  from 
Kehewin  school  and  one  hundred  and  nine  from  Onchaminahos.  Between  five  and 
ten  students  did  not  respond  to  any  given  item.  For  some  questions  the  non- 
responses would  rise  as  high  as  forty  to  fifty.  Usually,  however,  these  would  be 
related  to  items  which  were  subject  specific  and  one-half  of  the  students  at 
Onchaminahos  would  not  be  involved  with  DEGEM  for  that  subject.  In  1989,  the 
number  of  students  who  commented  was  significantly  lower,  with  53  students, 
grades  three  to  eight,  from  Kehewin  and  32  students,  grades  three  to  seven,  from 
Onchaminahos  responding.  The  grade  two  and  three  students  were  given  a 
questionnaire  where  the  agree-disagree  continuum  was  represented  by  happy- 
unhappy  faces.  These  results  were  later  transcribed  into  numerical  values.  For 
children  in  grades  four  and  up  the  responses  were  recorded  numerically.  In  1988, 
this  was  followed  by  a structured  interview  with  a small  sample  of  students  (n=33) 
selected  from  grades  two  through  six. 

Generally,  the  student  responses  were  quite  positive.  It  is  noteworthy,  however, 
that  on  questions  where  teachers  or  others  rated  the  students  as  having  difficulty  or 
disliking  an  aspect  of  the  system,  the  students  still  gave  the  system  high  marks.  Of 
the  forty-four  questions  asked  of  students  (1988)  , there  was  a statistically 
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significant  difference  between  the  two  groups  on  fourteen  items  or  approximately 
thirty-one  percent  (p<0.05).  In  1989,  there  was  a significant  difference  between  the 
schools  on  sixteen  questions  or  approximately  thirty-six  percent  of  the  total  items. 

The  students  were  first  given  a series  of  general  questions  regarding  DEGEM. 
They  indicated  that  DEGEM  helps  them  learn  and  that  it  made  their  job  at  school 
easier.  Both  groups  wanted  to  see  the  system  used  in  other  core  and 
complementary  subjects.  Although  both  groups  saw  the  computer  as  helping  them 
learn  faster,  there  was  a significant  difference  in  favor  of  the  Kehewin  students. 
When  asked  whether  the  DEGEM  computer  did  everything  they  thought  it  would, 
the  answers  were  positive  but  lower  than  most  other  responses.  Both  groups 
agreed  that  there  should  be  more  DEGEM  computers.  However,  the  Kehewin 
students  were  much  more  positive  than  the  Onchaminahos  students,  whose 
agreement  with  this  statement  dropped  slightly  over  time. 

Students  were  asked  specific  questions  regarding  their  own  perceptions  of 
achievement.  Both  groups  indicated  mathematics  was  easier  since  DEGEM  arrived, 
with  the  Onchaminahos  students'  responses  becoming  more  positive  over  time.  As 
well,  there  was  agreement  that  reading  was  easier,  with  Kehewin  students  agreeing 
more  strongly  with  this  statement.  According  to  the  students,  both  mathematics  and 
reading  are  more  fun  on  the  DEGEM  computer. 

The  balance  of  the  questionnaire  offered  a range  of  general  questions.  Both  groups 
enjoyed  working  on  the  system.  There  was  agreement  that  the  system  was  easy  to 
use,  made  learning  fun,  and  that  they  would  like  to  work  on  the  computer  more 
often.  Again,  on  all  three  questions  there  was  a statistically  significant  difference 
between  groups  with  the  Kehewin  group  scoring  higher.  There  was  agreement  that 
the  DEGEM  computer  helps  students  learn  mathematics  better  than  the  mathematics 
class  alone.  As  well,  students  agreed,  although  less  strongly,  that  the  DEGEM 
computer  helps  them  learn  language  arts  better  than  the  language  arts  class  alone. 
When  asked  to  respond  to,  "What  I learn  on  the  DEGEM  computer  helps  me  learn 
in  the  classroom.",  the  respondents  agreed,  with  the  Onchaminahos  students' 
response  becoming  more  positive  over  time.  In  response  to,  "The  DEGEM 
computer  helps  me  learn  my  school  work."  the  Onchaminahos  students'  became 
slightly  more  positive  over  time.  When  asked  if  they  needed  help  from  their  teacher 
when  on  the  system,  Kehewin  students  indicated  that  this  was  generally  not  the 
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case.  The  response  of  Onchaminahos  students'  suggested  that,  over  time,  the 
students  required  more  teacher  assistance  while  working  on  the  DEGEM  computer. 
Students  from  Kehewin  disagreed  with,  "I  like  learning  in  the  classroom  better  than 
in  the  DEGEM  computer  room.",  although  the  Onchaminahos  students'.were  more 
neutral  on  this  issue.  Both  groups  indicated  they  thought  the  computer  helps  them 
understand  their  mistakes,  tells  them  when  they  are  wrong,  and  how  they  are 
doing.  Few  admitted  to  wasting  time  when  on  the  system  or  that  it  was  hard  to 
learn  how  to  use  the  system.  When  asked  if  they  got  along  better  with  their  teacher 
since  using  DEGEM,  they  responded  in  the  mild  affirmative.  When  asked,  if  when 
they  got  stuck  did  their  teacher  help  them,  both  groups  agreed.  Both  groups 
indicated  that  when  they  made  a mistake  on  the  computer  they  needed  their  teacher 
to  help  them  understand  where  they  went  wrong.  When  asked  if  ten  minutes  was 
too  short,  they  agreed.  On  the  issue  of  whether  they  were  confused  when  the 
computer  asked  them  to  enter  an  answer  right  to  left  (e.g.,  DEGEM  requires  the 
number  "15"  to  be  entered  "5"  then  "1"),  their  responses  were  neutral  to 
affirmative,  with  Onchaminahos  students  agreeing  much  more  strongly  than 
Kehewin  students. 

Both  groups  indicated  that  when  they  get  an  answer  correct  the  computer  lets  them 
know  in  an  interesting  way.  Kehewin  students  agreed  that  the  graphics  are  as  good 
as  other  computers,  with  Onchaminahos  students  being  less  positive.  There  was 
agreement  that  the  DEGEM  system  asks  questions  differently  than  textbooks,  yet 
they  indicated  that  this  did  not  confuse  them. 


Table  12 

Mean  Values  of  Questions  on  The  Student  Questionnaire 

1988  1989 


Keh 

One 

Keh 

One 

The  DEGEM  computer  helps  me  learn.  3.5 

3.2 

3.3 

3.2 

The  DEGEM  computer  makes  my  job  at  school 
easier.  3.2 

2.9 

3.2 

2.8 

I would  like  to  see  the  DEGEM  computer  used  in 
other  core  subjects  like  social  studies  and  science.  3.2 

3.0 

3.5 

3.0 

69 


TABLE  12  Continued 

Mean  Values  of  Questions  on  The  Student  Questionnaire 

1988 


Keh 

One 

4. 

I would  like  to  see  the  DEGEM  computer  used  in 

complementary  subject  areas  like  art  and  music. 

2.9 

3.2 

5. 

The  DEGEM  computer  helps  me  learn  faster. 

3.2 

2.4 

6. 

The  DEGEM  computer  bugs  me. 

0.8 

1.5 

7. 

The  DEGEM  computer  does  everything  I thought 

it  would. 

2.4 

2.3 

8. 

The  DEGEM  computer  is  great! 

3.5 

3.3 

9. 

We  should  have  more  DEGEM  computers  in  our 

school. 

3.4 

3.1 

10. 

The  DEGEM  computer  works  the  way  it  is 

supposed  to  all  the  time. 

3.1 

2.3 

11. 

I think  the  DEGEM  computer  wastes  time. 

1.3 

1.5 

12. 

Math  is  easier  for  me  since  I have  been  using  the 

DEGEM  computer. 

3.2 

2.8 

13. 

Reading  is  easier  for  me  since  I have  been  using 

the  DEGEM  computer. 

3.2 

2.9 

14. 

Math  is  more  fun  when  I use  the  DEGEM 

computer. 

3.1 

2.8 

15. 

Reading  is  more  fun  when  I get  to  use  the 

DEGEM  computer. 

3.1 

2.8 

16. 

I like  to  work  on  the  DEGEM  computer. 

3.6 

3.3 

I find  the  DEGEM  computer  computer  easy  to 

17. 

use. 

3.5 

3.0 

18. 

I find  learning  on  the  DEGEM  computer  fun. 

3.5 

3.1 

19. 

I would  like  to  work  on  the  DEGEM  computer 

more  often. 

3.6 

3.1 

20. 

The  DEGEM  computer  helps  me  learn  math  better 

than  my  math  class  alone. 

3.1 

2.9 

21. 

The  DEGEM  computer  helps  me  learn  language 

arts  better  than  my  language  arts  class  alone. 

2.9 

3.1 

1989 

Keh  One 

3.4  2.9 

3.1  2.8 

1.9  1.4 

2.5  2.0 

3.5  2.8 

3.5  2.9 

2.5  2.4 

1.4  1.8 

3.1  3.3 

2.9  2.8 

3.5  3.2 

3.3  2.8 

3.3  3.3 

3.5  3.3 

3.4  3.0 

3.6  2.9 

3.2  3.1 

2.6 


2.8 
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TABLE  12  Continued 

Mean  Values  of  Questions  on  The  Student  Questionnaire 

1988  1989 


Keh 

One 

Keh 

One 

22.  The  DEGEM  computer  helps  me  leam  my  school 
work. 

3.3 

2.8 

3.3 

3.0 

23.  What  I leam  on  the  DEGEM  computer  helps  me 
leam  in  the  classroom. 

3.1 

2.6 

3.2 

3.2 

24.  I need  help  from  my  teacher  when  I use  the 
DEGEM  computer. 

1.7 

1.5 

2.1 

2.2 

25.  I like  learning  in  the  classroom  better  than  in  the 
DEGEM  computer  room. 

1.4 

1.4 

1.5 

2.2 

26.  I find  the  questions  on  the  DEGEM  computer  too 
hard. 

1.3 

1.5 

2.1 

1.4 

27 . The  DEGEM  computer  helps  me  understand  my 
mistakes. 

3.2 

3.1 

3.1 

2.8 

28.  The  DEGEM  computer  tells  me  when  I'm  wrong.  3.3 

3.1 

3.4 

3.1 

29.  The  DEGEM  computer  tells  me  how  I’m  doing. 

3.5 

3.4 

3.3 

3.2 

30.  When  I'm  supposed  to  be  working  on  the 
DEGEM  computer  I waste  time. 

1.6 

1.5 

2.0 

2.0 

31.  It  was  hard  for  me  to  leam  how  to  use  the 
DEGEM  computer. 

1.6 

1.7 

-1.4 

1.7 

32.  I understand  why  my  teacher  has  me  using  the 
DEGEM  computer. 

3.0 

2.9 

3.3 

2.8 

33.  I get  along  better  with  my  teacher  since  I have 
been  working  on  the  DEGEM  computer. 

2.6 

2.7 

2.8 

2.6 

34.  I leam  better  in  the  classroom  since  I have  been 
using  the  DEGEM  computer. 

3.0 

2.6 

3.2 

2.7 

35.  When  I get  stuck  on  the  DEGEM  computer  my 
teacher  helps  me. 

3.3 

2.9 

2.9 

3.1 

36.  When  I make  a mistake  on  the  DEGEM  computer 
I need  my  teacher  to  help  me  understand  where  I 
went  wrong. 

2.8 

2.4 

2.8 

2.8 
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TABLE  12  Continued 

Mean  Values  of  Questions  on  The  Student  Questionnaire 

1988  1989 

Keh  One  Keh  One 


37. 

Ten  minutes  is  too  short  a time  to  be  on  the 
computer.  2.7 

2.6 

3.0 

2.9 

38. 

In  math,  sometimes  the  answers  are  entered  "left 
to  right"  and  sometimes  they  are  entered  "right  to 
left".  This  confuses  me.  2.5 

2.0 

2.6 

3.0 

39. 

When  I am  using  the  DEGEM  computer  I fmd  it 
doesn't  work  properly.  1.7 

1.9 

2.0 

2.3 

40. 

When  I get  a question  right  the  DEGEM  computer 
lets  me  know  in  an  interesting  way.  3.3 

3.2 

2.9 

3.2 

41. 

The  DEGEM  computer  graphics  are  as  good  as 
other  computers.  3 . 1 

2.8 

3.4 

2.7 

42. 

The  fact  that  the  DEGEM  computer  terminal 
screens  use  green  only,  rather  than  several  colors, 
disappoints  me.  2.3 

1.7 

2.5 

3.1 

43. 

The  way  the  DEGEM  computer  asks  questions 
about  school  subjects  is  different  than  the  way  the 
textbooks  ask  questions.  3.1 

2.8 

2.7 

3.2 

44. 

The  way  the  DEGEM  computer  asks  questions 
confuses  me.  1.8 

1.7 

2.4 

2.0 

In  1988,  there  were  a number  of  students  who  wrote  eomments,  as  they  were 
invited  to  do  so;  in  1989,  fewer  students  wrote  eomments.  The  eomments  were 
generally  quite  short  and  revolved  around  several  themes  whieh  will  be  reported  by 
division.  In  1988,  division  one  students  pereeived  the  eomputer  as  helping  them 
learn  (35);  that  they  liked  using  it  for  the  subjeet(s)  they  had  (31);  that  it  was  fun 
(11);  that  they  like  typing  (9).  In  1989,  the  division  one  students  commented  that 
the  computer  helped  them  learn  (9),  that  they  liked  using  the  computer  for  math  (5), 
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and  that  it  was  fun  (5).  For  division  two  students  (1988),  there  were  similarities  in 
that  DEGEM  was  perceived  as  helping  them  learn  (35)  and  that  it  was  fun  to  use 
(25);  however,  they  also  commented  that  it  "breaks  down  a lot"  (23)  and  they 
would  like  a color  screen  (15).  In  1989  this  student  group  indicated  that  the 
computer  helps  them  learn  (13);  it  keeps  them  busy  (6);  they  enjoy  the  games  (5); 
and  they  like  the  feedback  (3).  This  older  group  was  more  critical  of  the  system 
and  commented  on  the  following:  sometimes  it  does  not  work  (8);  would  like  more 
color  and  graphics  (6);  it  does  not  give  feedback  when  one  or  two  errors  are  made 
(6);  and  that  working  on  the  computer  was  boring  (5).  They  spontaneously 
commented  that  they  would  like  more  time  on  the  computer  (6),  they  would  like  to 
use  it  for  more  subjects  (1 1),  and  they  would  like  more  games  (4). 

The  results  of  the  structured  interviews  (1988)  were  similar  in  that  any  general 
comments  made  were  brief.  When  asked  if  they  liked  working  on  the  DEGEM 
computer,  thirty  answered  "yes",  and  three  answered  "no".  "It  helps  you  learn", 
and  "It's  fun",  were  the  two  most  popular  responses.  When  asked  to,  "Pick  a 
number  that  tells  how  much  you  like  the  DEGEM  computer"  the  responses  ranged 
from  one  (n=2)  to  ten  (n=17)  with  a mean  of  8.2.  When  asked  how  often  they 
used  the  system  each  week,  about  one-half  were  able  to  indicate  the  frequency  with 
any  degree  of  accuracy.  When  asked  if  they  had  enough  time  on  the  computer,  ten 
indicated  "no"  while  twenty-three  indicated  "yes".  When  asked  what  the  best  thing 
about  the  DEGEM  system  is,  students  indicated:  that  it  helped  them  learn  (7);  it  was 
fun  (5);  it  was  easy  to  use  (4);  and  they  liked  typing  (4).  When  asked  what  the 
worst  thing  was,  they  said:  when  I get  the  questions  wrong  (6);  when  it  breaks  (4); 
the  subject  I take  (e.g.,  mathematics,  reading)  (4).  Finally,  the  students  were 
asked,  "If  you  could  make  one  wish  about  the  computer,  what  would  it  be?"  The 
answers  were  interesting  and  creative  - play  games  (7);  spend  more  time  on  it  (5); 
that  I would  get  all  right  (3);  that  it  would  not  break  (3);  have  color  (2);  it  would  talk 
(2);  it  would  play  rock  music  (1);  that  I could  have  my  own  (1). 

The  final  area  that  was  investigated,  in  1988, was  time  on  task.  The  procedure  used 
to  gather  data  involved  selecting  a number  of  students  at  random.  They  were 
observed  both  in  the  classroom  and  on  the  DEGEM  system.  Complete 
observations  were  conducted  on  twenty-two  students  at  Kehewin  and  twenty 
students  at  Onchaminahos.  Four  students  were  observed  at  each  grade  level 
(grades  2-7  inclusive  at  Kehewin;  grades  2-6  inclusive  at  Onchaminahos).  Two 
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students  were  dropped  from  the  sample  at  Kehewin  due  to  incomplete  data.  The 
procedure  involved  observing  a student  in  the  classroom  and  then  observing  the 
same  student  in  the  same  subject  area  on  the  DEGEM  system.  Observations  were 
made  at  10-second  intervals  for  five  minutes  (thirty  observations)  in  the  classroom, 
and  the  same  measures  were  made  on  the  DEGEM  system  (total  observations  were 
sixty).  Observations  were  made  by  two  individuals  who  were  trained  to  using  a 4- 
point  rating  scale,  as  follows: 

Score  Description 

1 Student  is  actively  engaged  in  learning  (i.e.,  there  is  some  observable 

component  which  indicates  engagement  - writing,  pushing  a button, 
talking  to  the  teacher). 

2 Student  is  oriented  toward  learning  (i.e.,  there  is  no  observable  evidence 

of  engagement,  but  the  student  is  oriented  toward  the  learning  activity  - 
looking  at  the  teacher,  looking  at  the  computer  screen). 

3 Student  is  disengaged  or  off  task  (i.e.,  there  is  some  observable 

component  which  indicates  off  task  behavior  - playing  with  a toy, 
reading  a book  when  should  be  writing  talking  to  classmates). 

4 This  option  was  to  be  used  if  the  observer  could  not  decide  on  a category; 

it  was  never  used. 

The  sixty  observations  for  each  student,  were  combined  to  yield  a total  of  ten 
observations  per  student,  (i.e.,  six  10-second  observations  were  averaged  to  yield 
one  observation  one  minute  in  length,  giving  a total  of  ten  observations). 
Therefore,  the  data  analysis  was  conducted  on  ten  measures;  five  "in-class",  and 
five  "on-DEGEM"  observations,  with  each  observation  representing  one  minute. 
The  data  were  analyzed  in  two  directions:  within  schools  - class  by  DEGEM;  and 
between  schools  - class  by  DEGEM. 

Within  schools,  each  minute  of  in-class  behavior  was  compared  with  each  minute 
of  on-DEGEM  behavior  for  the  total  number  of  subjects  in  the  school.  As  well,  the 
average  of  the  in-class  observations  (five  minutes  all  subjects  combined)  was 
compared  to  the  average  of  the  on-DEGEM  behaviors  (five  minutes  all  subjects 
combined).  A single  factor  (time  on  task)  analysis  of  variance  (ANOVA)  with 
repeated  measures  (twelve  observations)  was  run  "within  schools".  There  was  no 
statistically  significant  difference  between  time  on  task  in-class  or  on-DEGEM  in 
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either  school.  Onchaminahos'  results  yielded  an  F Ratio  of  1.24  with  a probability 
of  0.28.  Kehewin’s  results  yielded  an  F Ratio  of  1.28  with  a probability  of  0.27. 

When  the  average  time  on  task  figures  for  in-class  and  on-DEGEM  were  compared 
for  each  school,  no  statistically-significant  results  were  indicated.  Onchaminahos’ 
results  showed  a mean  difference  of  0.10  (little  difference,  but  favoring  on- 
DEGEM  time  on  task),  yielding  a probability  of  0.92.  Kehewin’s  results  showed  a 
mean  difference  of  -1.33  (a  small  difference,  but  favoring  in-class),  yielding  a 
probability  of  0.20. 

As  can  be  seen  from  all  the  results,  there  are  no  statistically  significant  differences 
between  in-class  and  on-DEGEM  time  on  task  behaviors  in  either  school.  There 
appears  to  be  a modest  trend  which  would  indicate  that  students  at  Onchaminahos 
are  on  task  more  when  using  DEGEM;  and  Kehewin  students  are  on  task  more 
when  in  class. 

Between  schools,  each  minute  of  in-class  behavior  in  one  school  was  compared 
with  the  same  minute  of  in-class  behavior  in  the  other  school.  The  same 
comparison  was  made  for  the  on-DEGEM  behavior.  As  well,  the  average  of  the  in- 
class  observations  (five  minutes  all  subjects  combined)  was  compared  to  the 
average  of  the  on  DEGEM  behaviors  (five  minutes  all  subjects  combined).  The 
results  are  reported  in  Table  13.  As  can  be  seen  from  the  results,  there  is  a 
statistically  significant  difference  between  schools  in  both  the  in-class  and  on- 
DEGEM  time  on  task  behaviors  (p<0.05).  In  both  cases  the  Onchaminahos  School 
obtained  higher  mean  scores.  When  considering  the  average  value  of  the  five 
minutes  of  observation,  the  Kehewin  students  appeared  to  be  on  task  more,  in  both 
the  in-class  and  on-DEGEM  settings. 
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Table  13 

Comparison  of  Time  on  Task  Bv  Minute  In-Class  and  On  DEGEM  Between 
ggbQQls 

Minutes  Means  Standard  Deviation  T Prob.  2-Tail 


In-Class 


One 

Keh 

One 

Keh 

1 

1.51 

1.75 

0.47 

0.51 

-1.6 

0.12 

2 

1.41 

1.80 

0.41 

0.60 

-2.5 

0.02 

3 

1.64 

1.87 

0.51 

0.59 

-1.41 

0.18 

4 

1.64 

2.10 

0.41 

0.63 

-2.8 

0.01 

5 

1.59 

2.01 

0.51 

0.68 

-2.3 

0.03 

Ave. 

1.56 

1.90 

0.31 

0.44 

-3.0 

0.01 

On-DEGEM 


One 

Keh 

One 

Keh 

1 

1.74 

1.73 

0.42 

0.47 

0.0 

1.00 

2 

1.69 

1.91 

0.49 

0.51 

-1.41 

0.16 

3 

1.69 

2.00 

0.42 

0.52 

-2.1 

0.04 

4 

1.59 

2.00 

0.23 

0.55 

-3.2 

0.00 

5 

1.58 

1.83 

0.39 

.047 

-1.8 

0.02 

Ave. 

1.66 

1.89 

0.22 

0.40 

-2.4 

0.02 

DISCUSSION:  Question  Seven 

The  qualitative  data  required  to  answer  this  question  were  gathered  on-site  from 
both  locations  through  questionnaires  and  structured  interviews.  The  students' 
responses  were  on  the  whole,  quite  positive,  with  the  Kehewin  students  being 
generally  more  positively  oriented  toward  the  system  and  the  Onchaminahos 
students'  responses  becoming  slightly  more  positive  over  time.  The  most  likely 
explanation  for  this  has  to  do  with  the  system  malfunction  at  Onchaminahos 
discussed  in  detail  in  Question  Four.  However,  the  1989  results  were  also 
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indicative  of  some  dissatisfaction  regarding  the  reliability  of  the  system  at  both 
sites.  The  division  one  students  were  less  critical  of  the  system  than  the  division 
two  students. 

Students  indicated  that  feedback  from  the  computer  was  given  in  an  interesting 
way,  yet  several  indicated  in  written  comments,  that  they  wished  there  was  more 
variety  in  how  the  computer  responded  to  them.  Further,  some  students  indicated 
that  the  computer's  feedback  sometimes  confused  them  and  that  they  would  like 
feedback  when  they  made  only  one  or  two  errors.  As  well,  teachers  expressed 
some  concern  over  the  way  feedback  was  provided.  Generally,  students  indicated 
that  the  system  was  easy  to  use  and  that  they  were  able  to  understand  the  system 
requirements  almost  immediately.  They  indicated  they  thought  they  learned  better 
with  CAI  then  in  the  classroom  alone,  yet  no  rationale  was  offered.  It  could  simply 
be  that  they  enjoy  the  opportunity  to  work  on  the  system  or  a change  from 
classroom  routine.  They  indicated  modest  improvement  in  their  relationships  with 
teachers.  However,  given  that  their  responses  to  other  questions  were  generally 
stronger,  one  would  not  expect  that  relationships  have  not  changed  much.  They 
also  indicated  that  when  they  were  stuck  they  needed  their  teacher  to  help  them  find 
where  they  made  the  mistake. 

General  comments  requesting  color  terminals,  or  more  games  or  game-type 
formats,  were  common.  As  well,  many  students  suggested  an  interest  in  using  the 
computer  in  other  subject  areas.  Therefore,  it  appears  that  students  enjoy  using  the 
system  but  have  some  areas  where  they  would  like  to  see  modifications.  However, 
a few  of  the  older  students  indicated  that  they  did  not  like  working  on  the  DEGEM 
computer  and  found  it  boring. 

The  time  on  task  data  used  a sample  of  forty-two  students  who  were  observed 
individually  for  five  minutes  in  the  classroom  and  five  minutes  on  DEGEM.  No 
statistically  significant  differences  were  found  between  time  on  task  in  the  class  and 
time  on  task  while  on  DEGEM.  Therefore  it  would  appear  that,  in  this  case,  the  use 
of  the  system  makes  little  difference  when  time  on  task  is  compared  to  the 
classroom. 
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Question  Eight  - To  what  extent  is  the  DEGEM  CAl  hardwarel software 
configuration  suitable  for  instruction  in  the  Alberta  context?  How  does  cost  per 
student-  instructional  hour  compare  with  conventional  teaching  and  other  hardware 
systems?  How  reliable  is  the  system  in  terms  of  downtime?  How  adequate  is 
vendor  support  in  terms  of  technical  training,  maintenance,  modifications  and 
updates?  Of  what  utility  are  the  authoring  and  support  components  of  the  system? 
How  flexible  and  adaptable  is  the  system  to  change? 

This  question  has  presented  itself  as  one  of  the  most  difficult  to  answer.  During  the 
first  round  of  data  collection  several  difficulties  were  encountered.  First,  accurate 
cost  figures  were  difficult  to  obtain  or  determine.  As  well,  no  records  had  been 
kept  with  respect  to  computer  down  time.  Consequently,  what  remains  were  the 
impressions  of  teachers,  students,  and  systems  managers.  Regardless,  data  which 
has  been  acquired  during  the  first  round  of  data  collection  will  be  reported.  Sources 
of  the  data  will  be  identified  as  appropriate.  The  questionnaires  provided  the 
perspectives  of  some  of  the  stakeholder  groups  in  a qualitative  format.  Quantifiable 
data  were  harder  to  collect  (e.g.,  on-going  costs,  number  of  users,  number  of  hours 
of  use).  When  the  interim  report  was  submitted  one  of  the  areas  that  was  discussed 
was  the  need  for  more  accurate  record  keeping  in  this  area;  however,  when  the 
second  round  of  data  collection  occurred,  it  was  discovered  that  no  information  that 
would  assist  in  answering  this  question  more  fully  had  been  recorded.  Table  14 
reports  the  responses  of  the  teachers  to  the  questions  which  relate  to  this  topic.  As 
was  stated  earlier,  the  Steering  Committee  indicated  that  given  the  number  of  staff 
changes  during  the  1988-1989  school  year,  the  systems  managers  and 
administration  would  be  given  the  teacher  questionnaire.  Therefore,  only  during 
the  1988  data  collection  are  there  any  specific  responses  from  systems  managers 
and  administrators. 

This  section  will  begin  with  a summary  of  the  qualitative  data.  With  respect  to  the 
issue  of  hardware  and  software,  several  questions  were  posed  to  teachers, 
administrators  and  systems  managers.  When  asked  if  the  mainframe  crashes 
frequently,  the  responses  were  in  the  strongly  disagree  to  disagree  range  in  1988, 
with  the  Kehewin  teachers’  responses  being  significantly  lower  (i.e.,  more 
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favorable).  The  systems  managers'  responses  were  similar  to  the  teachers' 
responses  in  their  own  schools.  In  1989,  the  Onchaminahos  teachers'  impressions 
were  slightly  more  favorable  than  in  1988.  When  asked  how  the  detachable 
keyboards  functioned,  in  1988  the  teachers  reported  that  they  worked  well,  with  the 
Kehewin  teachers'  results  being  more  favorable.  It  is  interesting  to  note  that  the 
systems  managers  indicated  that  the  keyboards  did  not  function  well.  In  1989  the 
Onchaminahos  teachers’  impressions  of  the  keyboards  changed  in  that  they  now  see 
them  as  not  functioning  well.  When  asked  the  same  question  regarding  the 
terminals,  the  same  trends  were  apparent  both  between  the  groups  in  1988  and 
across  time  at  Onchaminahos  School.  In  1988  when  administrators  and  systems 
managers  were  asked  if  they  thought  DEGEM  hardware  often  requires 
maintenance,  there  was  a large  difference  between  their  answers,.  The 
administrators  indicated  that  very  litde  maintenance  was  required,  while  the  systems 
managers  indicated  that  the  system  often  requires  maintenance. 

In  1988,  when  the  teachers  and  systems  managers  were  asked  if  the  software 
appeared  to  be  outdated,  there  was  general  disagreement  with  the  statement. 
However,  there  was  a significant  difference  between  the  two  teacher  groups,  again 
with  Kehewin  teachers  being  more  supportive.  In  1989,  the  Onchaminahos 
teachers  indicated  that  they  now  agree  with  that  statement.  With  respect  to  being 
unable  to  run  language  arts  and  mathematics  at  the  same  time,  the  groups’ 
responses  were  generally  neutral  in  1988.  In  1989,  teachers  report  that  it  is  a 
disadvantage. 

■ u 

When  in  1988,  teachers  were  asked  if  there  were  language  arts  or  mathematics 
topics  they  wished  the  system  dealt  with,  their  responses  were  generally  neutral  to 
slight  agreement.  In  1989  there  is  now  definite  agreement  with  the  concept.  In 
both  years  the  teachers  have  indicated  that  additional  topics  should  be  added  at  some 
point.  The  systems  managers  indicated  the  program  may  allow  for  the  addition  of 
new  topics,  but  when  asked  if  it  would  be  easy  to  do,  they  responded  with  strongly 
disagree.  When  asked  if  the  system  provided  adequate  reinforcement  for  students, 
there  was  a significant  difference  between  the  teacher  groups,  with  the  Kehewin 
response  being  agree  to  strongly  agree  and  the  Onchaminahos  group  being 
generally  neutral.  Since  that  time  the  Onchaminahos  teachers  have  moved  from  a 
neutral  position  to  a disagree  position.  The  same  significant  difference  was  found 
(1988)  when  asked  if  the  green  screen  (versus  color)  affects  student  motivation. 
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Again,  in  1989  the  Onchaminahos  group  moved  closer  to  the  disagree  end  of  the 
continuum.  When  asked  if  the  graphics  were  as  good  as  the  graphics  on  other 
computers,  both  groups  agreed  that  this  was  not  the  case  (1988),  and  in  1989  there 
is  less  satisfaction  with  the  lack  of  graphics. 

The  systems  managers  (1988),  when  asked  if  they  frequently  found  errors  in  the 
software,  indicated  not  often.  They  also  indicated  that  they  were  able  to  handle 
most  of  the  system's  problems.  They  suggested  that  they  spend  a lot  of  time  fixing 
the  system's  problems  and  that  vendor  support  for  software  is  better  than  the 
support  for  hardware;  however,  support  for  trouble  shooting  was  seen  as  the 
strongest  of  the  three  areas.  There  was  some  question  whether  the  supporting 
documentation  for  the  system  was  adequate  and  both  indicated  that  they  do  not  use 
the  authoring  features  of  the  program. 

Only  three  questions  were  directed  to  students  regarding  the  reliability  and 
technology  of  the  DEGEM  system.  "When  I am  using  the  DEGEM  computer,  I find 
it  doesn't  work  properly"  yielded  results  in  the  disagree  to  neutral  range  (for 
Onchaminahos  and  Kehewin  respectively,  1988).  In  1989  the  responses  were 
neutral  to  slightly  agree.  When  asked  if  the  system  works  the  way  it  is  supposed  to 
all  the  time,  in  1988  Kehewin  students  indicated  general  agreement  while  the 
Onchaminahos  students  tended  toward  neutrality.  However,  in  1989  both  groups 
responded  to  this  statement  in  the  neutral  to  agree  range.  There  was  a significant 
difference  in  the  response  to,  "The  fact  that  the  DEGEM  computer  terminal  screen 
uses  green  only  rather  than  several  colors  disappoints  me."  with  Kehewin  students 
expressing  more  disappointment.  The  1989  results  show  the  Onchaminahos 
students  expressing  greater  disappointment. 


Table  14 

Mean  Values  of  Software  and  Hardware  Questions  - Teacher  Questionnaire 


1988 

1989 

Keh 

One 

One 

113. 

There  are  topics  in  language  arts  I wish  the  DEGEM 
system  dealt  with. 

2.3 

1.8 

3.3 

114. 

There  are  topics  in  mathematics  I wish  the  DEGEM 
system  dealt  with. 

2.3 

2.0 

3.1 

115. 

Additional  topics  should  be  included  in  the  DEGEM 
system  at  some  point. 

2.9 

2.3 

2.8 

116. 

My  students  move  more  quickly  through  the  topics 
than  I expected. 

2.0 

1.8 

1.5 

117. 

When  I am  using  the  DEGEM  computer  I find  it 
does  not  work  properly. 

1.3 

1.9 

2.0 

118. 

When  I take  my  class  to  work  on  the  DEGEM 
computer  I find  the  whole  system  (mainframe 
computer)  is  not  working. 

0.4 

1.6 

1.4 

119. 

When  I take  my  class  to  work  on  the  DEGEM 
computer  I find  the  detachable  keyboard  is  not 
working. 

0.8 

1.9 

2.6 

120. 

When  I take  my  class  to  work  on  the  DEGEM 

computer  I find  the  student  terminal  (including  the 
keys)  is  not  working. 

1.1 

2.1 

2.4 

121. 

The  DEGEM  software  appears  to  be  outdated. 

0.7 

1.4 

2.9 

122. 

Being  unable  to  run  the  language  arts  program  and 

the  math  program  at  the  same  time  is  a disadvantage.  2.3 

2.0 

3.1 

123. 

The  DEGEM  computer  provides  adequate 
reinforcement  for  my  students. 

3.6 

2.0 

1.1 

124. 

The  DEGEM  system  graphics  are  as  good  as  other 
computers. 

1.4 

1.3 

0.8 

125. 

The  fact  that  the  terminal  screens  use  green  only, 
rather  than  several  colors,  does  not  affect  my 
students'  motivation. 

3.2 

1.6 

1.1 
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In  1988,  several  attempts  were  made  to  obtain  data  regarding  the  initial  and  ongoing 
costs  of  the  system.  Requests  were  made  at  several  levels  - school,  band  council, 
government,  and  Innovative  Technologies  in  Education  Inc.,  the  supplier.  Indian 
and  Northern  Affairs  Canada  was  able  to  supply  the  initial  system  costs  (dated  on 
or  about  October  1986).  They  are  outlined  in  Table  15: 


Table  15 

Initial  DEGEM  System  Costs  in  Canadian  Dollars 

Central  Blue  Quills  System  (currently  at  Onchaminahos  School) 


Hardware  (24  terminals) 

$ 

95,000.00 

Software 

$ 

8,000.00 

Courseware 

$ 

33,500.00 

Programming  Languages 

$ 

5,000.00 

Authoring  Programs 

$ 

6,500.00 

Shipping  and  Installation 

.5.000,00 

Sub-Total 


$ 153,000.00 


Satellite  System  (currently  at  Kehewin  School) 


Hardware  (24  terminals) 

Software 

Courseware 

Shipping  and  Installation 


$ 95,000.00 
$ 8,000.00 
$ 25,000.00 
$ 5.000.00 

Sub-Total 


$ 133,000.00 


Services 


Teacher  Training 
Technicians  Training 
Systems  Manager  Course 
Spare  Parts 


$ 5,000.00 

$ 5,000.00 

$ 4,000.00 

$ 18.000.00 

Sub-Total 


$ 32.000.00 


Total  Project  $ 318,000.00 
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A request  was  made  of  Innovative  Technologies  in  Education  Inc.  to  supply  current 
prices  for  an  identical  system.  A 1988  price  list  was  received;  the  only  listing  for  a 
system  which  supplies  24  DEGEM  terminals  was  the  EX-48.  The  cost  for  this 
system  is  $75  500.00  U.S.  This  does  not  include  the  DEGEM  keyboards  which 
are  $158.00  U.S.  each.  The  prices  appear  to  be: 


US  Dollars  Canadian  Dollars  (@122) 


Hardware  (24  terminals) 

$ 

75,500.00 

$ 

92,110.00 

Keyboards 

$ 

3,792.00 

$ 

4,626.24 

Line  Voltage  Regulator 

$ 

4,100.00 

$ 

5,002.00 

Spare  Parts  Kit 

1. 

24mm 

1. 

29.280.QQ 

Total 

$ 

107,392.00 

$ 

131,018.24 

The  reason  a current  price  list  was  requested  was  there  was  some  indication  in  1988 
that  prices  had  declined  recently.  Initially,  it  appears  that  the  terminal  prices  have 
dropped;  however,  when  the  costs  of  the  keyboards  are  included  the  price  is  about 
the  same.  Spare  parts  have  increased  and  there  is  no  indication  of  prices  regarding 
software,  courseware,  inservicing,  or  installation.  Therefore,  it  is  not  possible  at 
this  time  to  accurately  compare  the  original  cost  to  a current  price  for  the  same 
system.  It  does  appear,  however,  that  costs  have  not  decreased,  and,  in  fact,  may 
have  increased  slightly  (with  respect  to  Canadian  dollar  equivalents).  In  1989, 
Tammy  Williams,  the  production  manager  with  Innovative  Technologies  in 
Education,  indicated  that  there  has  been  no  price  change  with  respect  to  me  DEGEM 
system  since  1988.  Therefore  the  only  impact  on  price  would  be  the  relative  value 
of  the  Canadian  dollar  to  the  U.  S.  dollar. 

There  was  some  indication  that  an  annual  service  contract  was  available  at  a cost  of 
between  $ 10,000.00  and  $ 12,000.00,  and  that  one  or  both  school(s)  has 
purchased  one.  However,  no  agency  was  able  to  supply  any  written  documentation 
to  that  effect.  Therefore,  the  exact  cost  is  not  known,  nor  is  it  known  whether  the 
amount  is  in  Canadian  or  U.  S.  dollars,  or  what  the  service  contract  includes.  If 
either  school  has  entered  into  this  sort  of  agreement  it  would  affect  the  cost 
effectiveness  figures. 

In  1988  the  schools  were  also  asked,  if  possible,  to  supply  costs  of  items  such  as 
paper,  back-up  tapes,  and  hidden  labor  costs  (i.e.,  time  spent  by  the  systems 
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manager  over  and  above  any  allocated  release  time).  Onchaminahos  School  was 
able  to  do  so,  and  estimated  an  annual  cost  of  about  $ 400.00  per  year  for  supplies. 
Labor  was  estimated  at  approximately  four  hours  per  week  for  forty  weeks.  Labor 
could  then  be  calculated  at  a 0.1  f.t.e.  teaching  position  or  about  $ 3,000.00 
(assuming  an  average  teaching  salary  of  $ 30,000.00).  This  would  yield  an  annual 
cost  of  about  $ 3,400.00.  However,  it  should  be  noted  that  no  records  appear  to 
have  been  kept;  therefore,  the  four-hour-per-week  figure  is  only  a rough  estimate, 
as  is  the  cost  of  materials.  This,  combined  with  the  fact  that  data  such  as  the 
number  of  student-hours  on  DEGEM  were  not  available,  results  in  an  inability  to 
calculate  figures  such  as  cost  per  student  instructional  hour.  As  well,  since  no 
known  official  records  are  kept,  no  "hard  data"  can  be  obtained  with  respect  to 
issues  such  as  downtime  or  maintenance  costs.  It  should  be  noted  that  no  similar 
information  was  supplied,  though  requested,  for  1989. 

In  1989  there  was  also  some  suggestion  that  Innovative  Technologies  in  Education 
was  no  longer  marketing  the  DEGEM  system  and  that  they  were  concentrating  on 
supplying  educational  software  for  IBM  computers.  In  discussion  with  Tammy 
Williams,  she  indicated  that  in  fact  there  was  every  intention  to  continue  to  market 
the  DEGEM  system.  She  also  indicated  that  in  terms  of  investment  of  time  there 
was  an  even  split  between  DEGEM  and  the  IBM  software. 


DISCUSSION:  Question  EIGHT 

Some  qualitative  data  and  less  quantitative  are  available  from  both  schools.  No 
records  appear  to  have  been  kept  (1988  or  1989)  with  respect  to  downtime,  vendor 
support,  or  hardware  repairs.  Therefore,  an  attempt  was  made  to  obtain  the  data 
from  the  stakeholders  in  a qualitative  format. 

The  two  teacher  groups  often  disagreed  on  the  reliability  of  the  system  in  1988,  due 
no  doubt,  to  their  differing  experiences  in  the  spring  of  1988.  Generally,  the 
impressions  were  that  the  system  functioned  well,  barring  the  one  major  crash  at 
Onchaminahos.  This  was,  in  spite  of  the  fact,  that  one  or  more  terminals  appeared 
to  always  be  out  of  service  in  each  school.  It  appears  that  the  mainframe  is  more 
reliable  than  the  terminals,  especially  the  key  pads.  It  is  also  noteworthy  that  the 
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teachers'  impressions  at  Onchaminahos,  with  respect  to  the  reliability  of  the 
hardware,  have  worsened  somewhat  over  time. 

From  the  students'  perspective,  from  1988  to  1989,  both  groups  indicate  that  the 
system  appears  to  be  less  reliable.  As  well  they  expressed  disappointment  at  the 
limitations  in  the  areas  of  graphics  and  the  use  of  color.  It  also  appears  that  even  in 
areas  where  the  Kehewin  students’  views  are  remaining  relatively  stable,  the 
Onchaminahos  students  are  expressing  greater  disappointment. 

The  authoring  component  appears  not  to  be  used  at  all,  and  according  to  the  two 
systems  managers,  the  system  is  not  particularly  flexible.  The  modification  and 
addition  of  programs  is  seen  as  quite  difficult. 

It  appears  that  the  initial  cost  of  the  two  twenty-four  terminal  systems  was  about 
$318,000.00  Canadian.  It  does  not  appear  that  the  replacement  costs  would  be 
significantly  different  at  this  time  ; although  not  all  figures  were  available  to  make 
direct  comparisons.  Few  other  cost  figures  were  available;  therefore,  cost  per 
student  instructional  hour  cannot  be  calculated. 

Reinhold  (1986)  discusses  the  issue  of  networked  microcomputer  systems  versus 
minicomputers  (such  as  DEGEM).  She  indicates  that  one  of  the  advantages  of 
having  networked  micros  is  that  when  the  main  system  is  off,  additional  software 
can  be  run  at  each  individual  station.  Such  is  not  the  case  with  DEGEM.  She  does 
indicate,  however,  that  at  least  two  vendors  of  other  mini- systems  are  providing 
that  capability.  With  respect  to  the  similarities  and  differences  between  minis  and 
networked  micros  she  indicates  that  both  use  a hard  drive  system  which  allows 
students  to  access  different  parts  of  the  program  simultaneously..  She  refers  to 
Randy  Lough  speaking  about  the  Dolphin  minicomputer  system.  He  says  minis, 
"provide  more  extensive  processing  power,  and  faster  response  time.  . .-users  can 
maintain  more  extensive  records."  (p.  43).  Tom  Seal,  discussing  networked  IBM's 
states,  "users  can  use  other  software  than  [that]  designed  for  the  ILS  [Integrated 
Learning  System]  . . . distributing  processing  power  among  several  users  [of 
micros]  . . . slows  students'  response  times  . . . networks  [are]  more  powerful  than 
mini-based  systems."  (p.  44). 
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Reinhold  goes  on  to  compare  four  ILS  systems,  DEGEM  being  one.  She  quotes  a 
price  of  $90,000.00  to  $100,000.00  (U.S.)  for  32  terminals,  a printer,  two 
software  packages,  installation,  two  days  of  teacher  training,  and  three  days  of 
systems  manager  training  (p.  47).  Prices  are  listed  for  the  other  systems,  but  as 
different  hardware  is  provided  direct  comparisons  are  difficult.  In  terms  of 
deployment  CCC  Microhost  has  1300  systems  in  use;  PlatoAVicat,  150;  Dolphin, 
75;  and  DEGEM,  300  (world- wide,  others  gave  U.  S.  figures). 

Should  the  reader  require  more  detail  on  networking  micros  the  May  1988  issue  of 
A±  magazine,  is  an  issue  totally  dedicated  to  the  networking  of  micro  computers 
(specifically  Apples). 

Some  of  these  issues  need  to  be  explored  in  more  detail,  at  some  point,  before  a 
determination  can  be  made  as  to  the  cost  effectiveness  of  the  DEGEM  system.  The 
magnitude  of  the  work  required,  renders  this  a study  of  its  own.  However,  the 
literature  reviewed  in  this  area  clearly  indicates  there  are  a wide  range  of  options  that 
require  exploration.  This  should  be  occur  prior  to  any  decisions  regarding 
choosing  a system  is  made. 
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Question  Nine  - Does  the  DEGEM  CAI  result  in  improvement  in  student 
achievement  in  reading  and  math  when  applied  in  special  education  contexts? 

To  answer  this  question,  a case  study  approach  was  used  as  no  consistent  data  were 
available  on  any  special  education  class.  That  is,  no  special  education  classes 
identified  as  falling  within  the  scope  of  this  study,  had  any  standard  assessment 
data  across  its  student  members.  To  meet  the  conditions  of  this  study  a class  had  to 
be  segregated  at  least  for  part  of  the  time,  the  students  had  to  be  between  grades  two 
and  seven  inclusive,  and  using  the  system.  During  the  1987-1988  school  year, 
only  one  class  met  this  requirement.  It  was  located  at  Onchaminahos  School. 
During  the  1988-1989  academic  year  this  school  reorganized  its  special  education 
classes.  As  a result  of  the  reorganization  there  were  no  special  education  classes 
which  fell  within  the  scope  of  this  study. 

The  format  used  here  will  be  similar  to  that  outlined  in  question  three.  One  student 
was  selected  from  the  special  education  class,  and  an  individual  diagnostic  profile 
was  written  for  the  student  from  both  the  DEGEM  and  the  CAT  perspectives.  The 
profiles  were  then  compared.  Only  one  student  was  chosen,  as  in  the  one  particular 
class  eligible  to  be  included  in  the  study,  only  five  students  had  completed  more 
than  twelve  sessions  on  the  DEGEM  system  (i.e.,  were  past  the  testing  phase).  Of 
those  five,  three  had  already  been  dropped  a level  on  at  least  one  topic. 

Cased 

The  student  selected  for  discussion  was  chosen  from  the  elementary  special 
education  class  (grades  3 to  6)  at  Onchaminahos  School.  J.L.  is  a female  student  in 
this  class  whose  current  grade  placement  is  grade  5. 

J.L.  wrote  the  Canadian  Achievement  Test,  Level  12  in  May  1988  (Grade  Level 
2.8).  Her  grade  equivalent  score  on  the  Total  Test  Battery  was  2.1.  For  the 
purpose  of  this  discussion,  only  the  mathematics  section  of  the  CAT  will  be 
considered.  The  CAT  results  for  J.L.  are  reported  in  Table  16: 
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Table  16 

J.L.*s  Canadian  Achievement  Test  Results  in  Mathematics 


Area 

Grade  Equivalent 

Stanine 

Percentile 

Math  Computation 

2.3 

1 

1 

Concepts  & Application 

0.6 

1 

1 

Total  Math 

1.5 

1 

1 

An  item  analysis  of  J.L.’s  CAT  scores  indicate  the  following: 

Addition  - good  overall  addition  skills  (8/10  correct) 

- difficulty  adding  two-digit  and  two  one-digit 
numbers,  horizontally 

- difficulty  adding  two  two-digit  numbers 

Subtraction  - difficulty  in  all  areas  (1/10  correct) 

- needs  work  in  basic  facts,  one-  and  two-digit 
numbers 

Multiplication  - difficulty  in  all  areas  (3/6  correct)  but  knows 

some  basic  facts 

- needs  work  in  basic  facts,  two  one-digit 
numbers 

- requires  help  multiplying  two-digit  and  one- 
digit numbers,  horizontal,  no  regrouping 

Division  - no  subtest  - not  part  of  the  Grade  2 curriculum 

Concepts  & Application  - difficulty  in  all  areas  (9/40  correct) 

- has  virtually  no  understanding  of  numeration, 
number  theory,  and  number  properties  (2/22 
correct) 

- some  knowledge  of  common  scales  and 
measurement/graphs  (5/13  correct) 

- beginning  understanding  of  geometry  i.e., 
triangle,  4-sided  figures  (2/5  correct) 
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The  CAT  results  indicate  that  J.L.  is  having  a great  deal  of  difficulty  in 
mathematics.  Her  computation  skills  are  at  an  early  grade  2 level;  she  is  competent 
in  addition,  but  has  virtually  no  subtraction  or  multiplication  skills.  Therefore,  her 
remedial  program  would  focus  on  building  her  subtraction  and  multiplication  skills. 
Her  understanding  of  mathematics  concepts  is  extremely  weak  (grade  equivalent 
0.6);  she  will  require  extensive  remediation  in  this  area.  There  may  be  some 
question  as  to  whether  or  not  J.L.  is  capable  of  fully  understanding  these  concepts 
as  she  is  in  grade  5 and  is  at  a mid-grade  1 level  in  number  theory.  It  would, 
therefore,  be  important  to  look  at  individual  assessment  results  to  realistically 
establish  achievable  academic  goals  for  this  student. 

The  DEGEM  arithmetic  printout,  dated  May  1988,  sorted  by  urgency,  places  J.L.  at 
the  top  of  the  list.  This  indicates  that  she  is  the  student  most  urgently  requiring 
teacher  assistance.  The  printout  indicated  that  she  had  completed  her  fourteenth 
session  (it  should  be  noted  that  the  first  twelve  sessions  are  designated  for  testing). 
At  the  present  time  she  is  working  at  an  overall  grade  equivalent  level  of  1.4. 
During  her  last  session,  J.L.  answered  forty-seven  of  sixty-three  questions 
correctly.  One  of  her  sixteen  errors  was  a time  out  error,  indicating  she  did  not 
respond  within  the  allotted  time.  Only  two  topics  are  open  to  her  and  the  remaining 
thirteen  are  closed  as  she  is  functioning  at  a grade  level  too  low  to  access  these 
topics.  She  has  been  dropped  a level  in  Topic  1 (14^1)  and  is  currently  working  on 
numerical  systems,  specifically,  filling  in  the  missing  number  (e.g.,  3,4,5,?,7).  In 
Topic  2,  addition  and  subtraction  to  twenty,  she  is  presently  working  on  addition, 
adding  one  to  numbers  from  zero  to  four. 

A diagnostic  summary  based  on  the  DEGEM  data  reads  as  follows:  J.L.  is  a grade 
5 student  currently  placed  in  special  education.  She  is  experiencing  a great  deal  of 
difficulty  in  mathematics.  She  has  just  begun  using  the  DEGEM  system  and  is 
functioning  at  a Grade  1.4  level.  There  are  only  two  topics  open  to  her;  numerical 
systems,  and  addition  and  subtraction  to  twenty,  and  she  is  already  experiencing 
difficulty.  She  has  already  been  dropped  a level  in  one  topic.  J.L.  is  currently 
working  on  very  basic  number  concepts  and  on  basic  addition  facts.  In  view  of  the 
elementary  level  she  is  working  at,  one  could  question  the  applicability  of  the 
DEGEM  system  for  her.  In  addition,  the  diagnostic  information  provided  by 
DEGEM  for  J.L.  is  very  limited.  For  a more  comprehensive  picture,  individual 
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assessment  information  would  be  required  and  would  likely  yield  more  suitable  and 
thorough  data. 

When  the  diagnostic  summaries  from  the  CAT  and  DEGEM  are  compared,  both 
systems  indicate  that  J.L.  is  having  extreme  difficulty  in  mathematics  and  is 
functioning  at  a mid  grade  1 level  (1.5  CAT;  1.4  DEGEM).  Both  systems  identify 
her  computation  skills  as  very  weak  and  her  understanding  of  numerical  concepts  as 
minimal  at  best.  However,  the  CAT  offers  a more  detailed  profile  of  J.L.'s  areas  of 
competencies  and  weaknesses,  giving  information  on  a wider  variety  of  topics. 
The  CAT  provides  information  on  all  areas  of  mathematics  computation,  and 
concepts  and  application,  whereas  the  DEGEM  system  provides  information  only 
on  the  two  topics  open  to  her.  Therefore,  the  CAT  diagnostic  system  provides  a 
more  complete  picture  of  J.L.'s  mathematics  skills  and  skill  deficits.  It  is, 
however,  important  to  note  that  neither  diagnostic  profile  provides  enough 
information  to  develop  a comprehensive  remediation  program.  It  is  advisable  that 
information  given  by  either  the  CAT  or  DEGEM  be  supplemented  with  other  data 
sources  (i.e.,  individual  assessment  results). 


DISCUSSION:  Question  Nine 

To  answer  this  question  adequately  is  most  difficult.  In  addition  to  the  major 
difficulty  of  having  the  special  education  classes  restructured,  two  other  factors 
affected  the  ability  to  answer  this  question:  one,  there  was  little  consistent  data, 
longitudinal  or  otherwise  on  the  special  education  students  who  fell  within  the 
parameters  of  this  study;  and  two,  the  special  education  students  who  were  included 
in  this  project  had  only  begun  to  use  the  DEGEM  CAI  system  (a  maximum  of  16 
sessions).  Therefore,  a case  study  format  was  applied,  using  only  one  student. 
The  conclusion  for  this  particular  individual  was  that  neither  the  DEGEM  nor  the 
CAT  analysis  should  be  used  in  isolation  to  develop  an  individual  educational 
program.  The  CAT  appeared  to  have  one  advantage  in  that  it  provided  information 
on  a broader  range  of  skills. 

A number  of  issues  surrounding  this  question  were  raised,  which  merit  discussion 
at  the  present  time.  First,  Is  DEGEM  CAI  an  appropriate  remedial  tool  for  severely 
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learning  disabled  or  mentally  handicapped  students?.  This  speaks  not  just  to  the 
ability  of  the  system  to  hold  a special  needs  student's  interest,  but  to  the 
applicability  and/or  utility  of  the  diagnostic  results.  This  leads  to  the  second 
question,  Does  the  information  provided  on  the  DEGEM  printout  significantly 
contribute  to  developing  an  educational  profile  of  a special  needs  student?.  It 
appears  that  these  students  have  such  specialized  needs  that  individual  assessments 
are  required;  therefore,  the  diagnostic  utility  of  the  DEGEM  system  is  extremely 
limited.  Consequently,  all  that  would  be  left  would  be  the  drill  activities.  This 
leads  to  the  last  question.  Is  the  content  presented  in  the  topics  on  DEGEM 
appropriate  for  special  needs  students?.  It  appears  that  the  answer  is  likely  "yes  and 
no".  A teacher  would  have  to  be  extremely  selective  of  the  topics  students  would 
be  allowed  to  use,  closing  some  topics  to  them.  Another  problem  might  be  that  the 
resulting  grade  equivalent  scores  may  be  too  low  to  allow  students  access  to  topics 
the  teacher  wants  them  to  use. 

Additional  data  are  required  to  answer  this  question  adequately.  It  was  unfortunate 
that  there  were  no  additional  data  which  could  be  collected  during  the  second  year 
of  the  study  due  to  the  reorganization  of  the  special  education  program  at 
Onchaminahos.  It  appears  reasonable  to  conclude  however  that  the  use  of  DEGEM 
for  special  needs  students  must  comprise  only  one  component  of  the  diagnostic  and 
programming  strategies  for  these  students. 


CHAPTER  THREE  - SUMMARY  AND  RECOMMENDATIONS 


Summary 

During  the  course  of  this  study,  from  mid- way  through  the  1987  -1988  school  year 
to  the  end  of  the  1988-1989  school  year,  the  use  of  the  DEGEM  computer  system  at 
Kehewin  and  Onchaminahos  schools  has  been  fraught  with  numerous  problems 
and  events.  Some  of  these  difficulties  were  internal  to  the  original  research  design; 
as  well,  there  were  a number  of  events  (e.g.,  high  staff  turn  over  and  student  cohort 
contamination)  throughout  this  time  frame  which  had  a serious  and  negative  impact 
on  the  results.  These  threats  to  validity  were  uncontrolable  and  definitely  impacted 
the  viability  of  this  study  and  subsequently,  the  validity  of  the  results.  From  the 
perspective  of  qualitative  data  collection,  information  exists  which  provides  a 
foundation  for  some  interesting  discussion  and  conclusions.  However,  when  the 
quantitative  data  are  considered  in  light  of  the  qualitative  data,  it  is  possible  to 
suggest  some  trends  and  to  pose  some  interesting  hypotheses.  This,  in  the  end,  is 
the  value  of  this  study,  which  is,  to  date,  the  only  longitudinal  study  regarding  the 
educational  impact  of  the  use  of  the  DEGEM  computer  system. 

With  respect  to  question  one,  which  is  concerned  with  the  instructional  benefit  of 
using  a combination  of  DEGEM  and  classroom  instruction  over  classroom 
instruction  alone,  the  data  are  inconclusive.  As  the  data  collected  to  answer  this 
question  were  to  be  primarily  quantitative,  and  were  greatly  contaminated,  no  valid 
conclusions  could  be  drawn.  However,  if  in  answering  this  question  the  qualitative 
data  are  also  considered,  some  statements  can  be  tendered.  Generally,  use  of  the 
DEGEM  computer  system  in  the  mathematics  area  was  perceived  as  educationally 
beneficial  by  both  students  and  teachers.  This  perception  remained  positive  over 
time  at  both  school  sites.  In  the  language  arts  area,  the  DEGEM  system  was  not 
perceived  to  be  as  educationally  advantageous  as  in  the  mathematics  area.  This 
perception  remained  constant  over  time.  This  difference  may  be  due,  at  least  in 
part,  to  the  apparently  poor  curriculum  fit  between  the  DEGEM  English  program 


92 


software,  which  emphasizes  skills,  and  the  Alberta  Language  Arts  Curriculum, 
which  emphasizes  process. 

Question  two,  which  was  concerned  with  the  appropriateness  of  the  DEGEM  CAI 
courseware  and  its  match  with  various  Alberta  Education  criteria  was  investigated 
by  Alberta  Education,  and  the  results  are  reported  in  Appendix  C. 

Question  three  dealt  with  the  diagnostic  capability  of  the  DEGEM  system,  which 
according  to  the  DEGEM  literature,  claims  to  be  an  effective  diagnostic  system. 
The  diagnostic  profiles  obtained  from  the  DEGEM  computer  were  compared  to  the 
diagnostic  profiles  obtained  from  the  Canadian  Achievement  Tests  on  a case  study 
basis.  This  analysis,  in  combination  with  results  of  the  teacher  interviews  and 
questionnaires,  indicated  that  it  is  difficult  for  teachers  to  fully  utilize  the  diagnostic 
printout  of  the  DEGEM  system.  In  addition,  there  was  a general  concensus  that  it 
is  not  possible  to  determine  the  actual  value  of  the  DEGEM  grade  equivalents 
relative  to  Alberta  standards.  Further,  the  diagnostic  capability  of  the  DEGEM 
system  is  limited  to  dealing  only  with  topics  which  are  open  to  the  student 
determined  by  the  student's  overall  grade  equivalent  score.  As  well,  the  system  is 
only  able  to  identify  weaknesses,  and  not  strengths.  Finally,  the  diagnostic  print- 
outs were  found  to  be  extremely  difficult  to  interpret.  In  part  this  is  due  to  the  fact 
that  the  user  would  have  to  refer  to  a number  of  manuals  to  master  the  very 
confusing  coding  system. 

Question  four  dealt  with  the  issue  of  the  quality  and  effectiveness  of  the  teacher 
inservice  regarding  DEGEM.  Through  the  collection  of  qualitative  data  via  teacher 
questionnaire,  it  was  clearly  evident  that  the  teachers  in  both  schools  responded 
positively  toward  the  inservice  provided,  by  both  the  DEGEM  personnel  and  their 
systems  managers.  However,  it  was  also  evident  that  the  teachers  perceived  a need 
for  further  inservice  in  order  to  fully  utilize  the  system.  Although  every  teacher 
could  use  the  system  (i.e.,  turn  it  on  and  off),  few  teachers  had  a thorough 
understanding  of  the  system's  potential  and  no  teachers  were  able  to  utilize  the 
diagnostic  functions  of  DEGEM. 

With  respect  to  question  five,  which  dealt  with  the  impact  of  DEGEM  CAI  on 
students'  thinking  skills,  the  quantitative  data  collected,  using  the  Test  of  Cognitive 
Skills,  were  subject  to  the  same  contamination  effects  discussed  previously  in 
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question  one.  Therefore,  it  is  not  possible  to  draw  any  conclusions  on  the  basis  of 
these  data.  However,  the  qualitative  data  collected  regarding  this  issue  suggests 
that  overall,  teachers  agreed  that  the  stronger  and  average  students  derived  the  most 
benefit  in  this  area  from  working  on  the  DEGEM  computer  system.  It  appeared  that 
the  weaker  students  benefitted  the  least.  Further,  teacher  perceptions  indicated  that 
time  on  the  DEGEM  system  may  enhance  the  student’s  ability  to  sequence 
information.  Although  teachers  indicated  that  they  had  noticed  an  improvement  in 
their  students’  thinking  skills  during  the  course  of  the  school  year,  they  were 
uncertain  that  these  differences  could  be  attributed  to  DEGEM. 

Data  collected  in  order  to  answer  question  six,  teacher  perceptions  of  the  usefulness 
of  DEGEM  and  the  ability  to  implement  the  system  as  intended,  was  exclusively 
qualitative.  Both  questionnaires  and  face-to-face  interviews  were  conducted  at  both 
sites.  Generally,  the  teachers  indicated  that  they  liked  the  DEGEM  system,  that  it 
was  able  to  diagnose  students’  areas  of  need,  that  it  provided  them  with  information 
for  individualized  instruction,  and  that  they  wanted  to  continue  to  use  it.  However, 
most  teachers  indicated  that  they  did  not  use  the  information  from  DEGEM  in  any 
specific  way,(e.g.,  individualizing  instruction).  Over  time,  the  popularity  of  the 
system  seemed  to  wane  at  Onchaminahos;  unfortunately,  there  was  no  concurrent 
data  available  for  Kehewin  with  respect  to  this  issue.  Teachers  consistently 
expressed  a desire  for  further  inservice,  which  was  not  provided.  Teachers  did  not 
perceive  that  the  use  of  the  DEGEM  computer  system  had  significantly  impacted 
their  teaching  style,  or  the  nature  or  quality  of  student-teacher  relationships. 
Although  they  generally  perceived  the  courseware  to  be  developmentally 
appropriate,  there  was  some  concern  expressed  regarding  cultural  appropriateness. 
A notable  trend  emerged  which  suggested  that  there  is  an  inverse  relationship 
between  teacher  knowledge  of  computers  and  positive  orientation  toward  DEGEM. 
That  is,  the  more  teachers  knew  about  computers,  the  less  favorably  they  perceived 
DEGEM. 

Question  seven  dealt  with  the  students’  perceptions  of  the  DEGEM  system.  As 
with  question  six,  qualitative  data  was  collected  at  both  sites,  using  a questionnaire 
and  face-to-face  interviews.  Generally,  the  students  were  positively  disposed 
toward  DEGEM,  with  the  Kehewin  students  being  more  positively  oriented  toward 
the  system  than  Onchaminahos  students.  However,  the  Onchaminahos  students’ 
response  became  more  positive  over  time.  Students  agreed  that  the  system  was 
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easy  to  use  and  that  they  were  able  to  understand  the  system  requirements  almost 
immediately.  Generally,  division  one  students  were  less  critical  of  the  system  than 
the  division  two  students.  A number  of  students  indicated  an  interest  in  using 
DEGEM  in  other  subject  areas.  Some  dissatisfaction  regarding  the  reliability  of  the 
system  hardware  was  evident  at  both  sites.  Comments  suggesting  using  color, 
graphics,  and  enhancing  game  formats  were  noted.  Classroom  observations  were 
conducted  in  1988  to  determine  whether  student  time  on  task  differed  in  the 
classroom  when  compared  to  time  on  task  when  on  the  DEGEM  system.  The  time 
on  task  data  indicated  that  there  was  no  difference  with  respect  to  time  on  task 
whether  the  students  were  on  the  system  or  in  the  classroom. 

Adequate  data  collection  in  response  to  question  eight,  regarding  the  suitability, 
reliability,  flexibility,  and  cost-efficiency  of  the  DEGEM  system,  was  difficult  to 
obtain.  However,  on  the  basis  of  the  data  collected,  some  statements  can  be  made. 
Teachers  generally  perceived  the  system  to  be  unreliable,  with  Onchaminahos 
teachers  being  less  positive  over  time.  Students  indicated  that  the  system  may  be 
unreliable  (i.e.,  the  keypads),  with  the  Onchaminahos  students  becoming  more 
critical  over  time.  Students  also  increasingly  expressed  a desire  for  the  utilization  of 
current  technology  with  this  system  (i.e., using  color  and  graphics).  Systems 
managers  indicated  that  the  authoring  component  is  not  being  used  at  all  and  that  the 
system  is  not  particularly  flexible.  Costs  for  the  system  have  remained  relatively 
stable  over  time.  Due  to  lack  of  information  at  the  school  level,  cost  per  student 
instructional  hour  could  not  be  calculated. 

Question  nine  dealt  with  the  application  of  the  DEGEM  system  to  the  special 
education  context.  Data  were  difficult  to  obtain  in  this  area,  as  there  was  no 
consistent  information  available  on  any  special  education  class.  As  a result,  a case 
study  approach,  using  only  one  student,  was  employed.  The  conclusion  for  this 
particular  individual  was  that  the  the  DEGEM  diagnostic  information  did  not 
provide  a comprehensive  enough  picture  to  form  the  basis  of  an  individual  program 
plan.  Therefore,  if  utilized,  this  information  should  be  used  within  the  context  of 
other  data.  However,  this  analysis  brought  some  important  issues  to  light.  First,  a 
question  as  to  the  suitability  of  the  DEGEM  system  as  an  appropriate  remedial  tool 
for  severely  learning  disabled  or  mentally  handicapped  students.  Second, 
information  provided  on  the  DEGEM  printout  may  not  significantly  contribute  to 
developing  an  educational  profile  of  a special  needs  student.  The  third  issue  had  to 
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do  with  the  suitability  of  the  topics  on  DEGEM  with  respect  to  special  needs 
students.  Additional  data  will  be  required  to  answer  this  question  adequately. 

In  conclusion,  use  of  the  DEGEM  computer  system  at  Kehewin  and  Onchaminahos 
schools  appears  to  have  been  positively  received  by  teachers,  administrators, 
parents,  and  students.  However,  teachers  have  indicated  that  they  require  extensive 
inservice  in  order  to  fully  utilize  the  DEGEM  system.  It  appears  that  the  diagnostic 
application  of  the  system  is  not  being  fully  utilized  and  that  further  inservice  and 
support  is  specifically  required  in  this  area.  As  well,  clarification  regarding  the  true 
meaning  of  the  grade  equivalents  and  an  understanding  of  the  coding  system  used 
on  the  DEGEM  printout  are  required.  Further,  it  seems  evident  that  the  DEGEM 
hardware  may  benefit  from  some  changes  which  would  make  it  more  consistent 
with  other  computer  systems  in  terms  of  reliability  and  flexibility.  This  would 
make  it  more  viable  for  the  educational  environment.  It  appears  that  there  is  a need 
for  the  technology  to  be  updated  and  expanded  in  order  to  provide  more  graphics, 
games,  and  color  features,  which  appeal  to  students.  Further  study  regarding  the 
utility  and  practicality  of  the  DEGEM  diagnostic  components  and  the  curriculum 
match  of  the  DEGEM  software  to  Alberta  curriculum  is  warranted. 

RECOMMENDATIONS 

Given  the  significant  difficulties  encountered  during  the  process  of  attempting  to 
collect  the  data  no  recommendation  can  be  made  as  to  whether  the  DEGEM  system 
should  or  should  not  be  considered  for  utilization  by  schools  in  Alberta.  However, 
some  recommendations  can  be  made  which  will  assist  any  school  considering  the 
use  of  this  system.  In  addition,  the  recommendations  could  be  used  to  guide 
further  quantitative  study,  should  it  be  undertaken. 

It  is  recommended  that: 

1 . Careful  consideration  be  given  to  filling  the  position  of  systems  manager.  It 
appears  that,  to  a very  large  degree,  the  successful  implementation  of  this 
system  in  a school  depends  on  the  skills,  background,  commitment,and 
inservice  training  of  the  systems  manager. 
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2 . Inservice  of  professional  staff  is  crucial  to  the  full  utilization  of  the  system  in  the 
school.  The  system  is  deceptively  simple,  in  that,  very  little  training  is  required 
to  operate  the  system;  however,  a significant  amount  of  inservice  is  required  to 
fully  utilize  the  system's  diagnostic  capabilities. 

3 . The  area  of  diagnostic  effectiveness  be  further  explored  prior  to  making  any 
recommendations  to  purchase  the  system.  This  study,  due  to  initial  design 
flaws  and  contaminated  quantitative  data  is  silent  in  this  area. 

4.  Prior  to  deciding  to  acquire  a system,  interested  parties  should  contact  Alberta 
Education  to  obtain  information  on  how  the  courseware  fits  with  Alberta's 
curriculum. 
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Appendix  A contains  copies  of  the  questionnaires  as  distributed  in  1988  at 
Onchaminahos  School.  The  only  difference  between  the  questionnaires  contained 
herein  and  those  distributed  at  Kehewin  school  was  that  the  word  "TOAM"  was 
substituted  for  the  word  "DEGEM"  throughout.  The  student  questionnaire  was 
further  modified  at  both  schools  for  the  division  one  students.  In  place  of  the  six 
point  rating  scale  (i.e.,  0 to  5)  there  were  a series  of  happy  faces,  frowny  faces. 
The  "Do  Not  Know"  response  was  replaced  with  a question  mark. 

The  student  questionnaire  distributed  in  1989  was  identical  to  those  distributed  in 
1988.  The  1989  teacher  questionnaire  was  modified  in  that  all  questions  which 
dealt  with  inservice  were  removed  (i.e.,  numbers  31  to  45  inclusive)  and  the 
balance  of  the  questionnaire  was  renumbered.  The  1988  form  of  the  administrator 
and  the  system  manager  questionnaire  were  omitted  from  the  1989  data  collection 
and  these  individuals  were  asked  to  complete  the  teacher  questionnaire. 

The  Teacher  Survey  Interview  Form  was  used  for  the  face-to-face  interviews  in 
both  1988  and  1989.  The  Student  Survey  Interview  Form  and  the  Parent  Survey 
Interview  Form  were  used  for  the  face-to-face  interviews  in  1988,  but  were  not 
used  in  1989. 
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THE  DEGEM  COMPUTER 
Student  Perspectives 


Read  aloud  with  the  students. 


On  the  following  pages  are  some  sentences  about  the  DEGEM  computer  in  your 
school.  Under  each  sentence  is  a rating  scale  with  the  numbers  0 to  5.  The 
meaning  of  each  number  is  as  follows: 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


After  you  have  read  the  sentence,  decide  which  number  best  describes  what  you 
think  about  the  sentence.  Fill  in  the  right  circle  on  the  answer  sheet  provided.  Try 
not  to  use  Number  2.  At  the  end,  there  is  a chance  for  you  to  write  what  you  think 
about  the  computer  if  you  want  to.  Thank  You. 

TO  THE  TEACHER: 

You  will  be  given  a machine  scoreable  answer  sheet  for  each  of  your  students. 
Please  complete  the  background  information  for  each  of  your  students.  Thank  you. 


PART  I - Background  Information 

Find  the  box  on  the  answer  sheet  labelled  "SEX";  indicate  your  gender  by  filling  in 
the  appropriate  circle. 

Find  the  box  labelled  "GRADE  OR  EDUC";  indicate  the  grade  you  are  in  by  filling 
in  the  appropriate  circle. 
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Find  the  boxes  under  "IDENTIFICATION  NUMBER": 

Box  "A"  - Using  the  same  rating  scale  as  mentioned  above  answer  the 
following:  "I  know  a lot  about  computers."  Fill  in  the  appropriate  circle. 

Find  the  box  labelled  "SPECIAL  CODES".  Put  your  "DEGEM  NUMBER"  in  this 
box  starting  with  Box  K,  one  digit  in  each  box. 

Box  "O"  - If  you  are  at  Kehewin  School,  fill  in  the  circle  "zero"  (0).  If  you 
are  at  Saddle  Lake  School,  fill  in  the  circle  "one"  (1). 

Box  "P"  - If  you  are  an  administrator,  fill  in  the  circle  "zero"  (0).  If  you  are  a 
teacher,  fill  in  the  circle  "one"  (1).  If  you  are  a system  manager,  fill  in  the 
circle  "two"  (2).  If  you  are  a student,  fill  in  the  circle  "three"  (3). 


PART  II  - General  Questions 


1 .  The  DEGEM  computer  helps  me  learn. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


2 .  The  DEGEM  computer  makes  my  job  at  school  easier. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


3 .  I would  like  to  see  the  DEGEM  computer  used  in  other  core  subjects  like 
social  studies  and  science. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 
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4 .  I would  like  to  see  the  DEGEM  computer  used  in  complementary  subject 
areas  like  art  and  music. 


0 12  3 4 

Strongly  Disagree  Neutral  Agree  Strongly 
Disagree  Agree 


5 

Do  Not 
Know 


5 . The  DEGEM  computer  helps  me  learn  faster. 


0 

Strongly 

Disagree 


1 

2 

3 

4 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

5 

Do  Not 
Know 


6 .  The  DEGEM  computer  bugs  me. 

0 1 2 
Strongly  Disagree  Neutral 
Disagree 


3 4 

Agree  Strongly 

Agree 


5 

Do  Not 
Know 


7 . The  DEGEM  computer  does  everything  I thought  it  would. 


0 

Strongly 

Disagree 

1 

Disagree 

2 

Neutral 

3 

Agree 

4 

Strongly 

Agree 

5 

Do  Not 
Know 

8 . The  DEGEM  computer  is  great! 

0 

Strongly 

Disagree 

1 

Disagree 

2 

Neutral 

3 

Agree 

4 

Strongly 

Agree 

5 

Do  Not 
Know 

9 . We  should  have  more  DEGEM  computers  in  our  school. 

0 

Strongly 

Disagree 

1 

Disagree 

2 

Neutral 

3 

Agree 

4 

Strongly 

Agree 

5 

Do  Not 
Know 

10.  The  DEGEM  computer  works  the  way  it  is  supposed  to  all  the  time. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 
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11.  I think  the  DEGEM  computer  wastes  time. 


0 

1 

2 

3 

4 

5 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Do  Not 
Know 

PART  III  - Specific  Questions 


Achievement 


12.  Math  is  easier  for  me  since  I have  been  using  the  DEGEM  computer. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


13.  Reading  is  easier  for  me  since  I have  been  using  the  DEGEM  computer. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


14.  Math  is  more  fun  when  I use  the  DEGEM  computer. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


1 5 .  Reading  is  more  fun  when  I get  to  use  the  DEGEM  computer. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 
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Student  Views 


16.  I like  to  work  on  the  DEGEM  computer. 


0 1 

2 

3 

4 

5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Do  Not 
Know 

17. 

I find  the  DEGEM  computer  easy  to  use. 

0 1 

2 

3 

4 

5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Do  Not 
Know 

18. 

I find  learning  on  the  DEGEM  computer  fun. 

0 1 

2 

3 

4 

5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Do  Not 
Know 

19. 

I would  like  to  work  on  the  DEGEM  computer  more  often. 

0 1 

2 

3 

4 

5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Do  Not 
Know 

20. 

The  DEGEM  computer  helps  me  learn  math  better  than  my  math  class  alone. 

0 1 

2 

3 

4 

5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Do  Not 
Know 

21. 

The  DEGEM  computer  helps  me  learn  language  arts 

better  than  my  language 

arts  class  alone. 

0 1 

2 

3 

4 

5 

Strongly  Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Do  Not 
Know 
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22.  The  DEGEM  computer  helps  me  learn  my  school  work. 

0 12  3 4 

Strongly  Disagree  Neutral  Agree  Strongly 
Disagree  Agree 


5 

Do  Not 
Know 


23. 


What  I learn  on  the  DEGEM  computer  helps  me  learn  in  the  classroom. 


0 

1 

2 

3 4 

5 

Strongly 

Disagree 

Neutral 

Agree  Strongly 

Do  Noi 

Disagree 

Agree 

Know 

24.  I need  help  from  my  teacher  when  I use  the  DEGEM  computer. 

0 12  3 4 

Strongly  Disagree  Neutral  Agree  Strongly 
Disagree  Agree 


5 

Do  Not 
Know 


25. 


I like  learning  in  the  classroom  better  than  in  the  DEGEM  computer  room. 


0 

1 

2 

3 

4 

5 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

Do  No 
Know 

26.  I find  the  questions  on  the  DEGEM  computer  too  hard. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


27 . The  DEGEM  computer  helps  me  understand  my  mistakes. 

0 12  3 4 

Strongly  Disagree  Neutral  Agree  Strongly 
Disagree  Agree 


5 

Do  Not 
Know 


2 8 . The  DEGEM  computer  tells  me  when  I'm  wrong. 

0 12  3 4 

Strongly  Disagree  Neutral  Agree  Strongly 
Disagree  Agree 


5 

Do  Not 
Know 
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29.  The  DEGEM  computer  tells  me  how  I'm  doing. 


0 

Strongly 

Disagree 


Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


30.  When  I'm  supposed  to  be  working  on  the  DEGEM  computer  I waste  time. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


31.  It  was  hard  for  me  to  learn  how  to  use  the  DEGEM  computer. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


32.  I understand  why  my  teacher  has  me  using  the  DEGEM  computer. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


33.  I get  along  better  with  my  teacher  since  I have  been  working  on  the  DEGEM 
computer. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


34.  I learn  better  in  the  classroom  since  I have  been  using  the  DEGEM  computer. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


35 .  When  I get  stuck  on  the  DEGEM  computer  my  teacher  helps  me. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 
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36.  When  I make  a mistake  on  the  DEGEM  computer,  I need  my  teacher  to  help 
me  understand  where  I went  wrong. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


37.  Ten  minutes  is  too  short  a time  to  be  on  the  DEGEM  computer. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


38.  In  math,  sometimes  the  answers  are  entered  "left  to  right"  and  sometimes  they 
are  entered  "right  to  left".  This  confuses  me. 


0 

Strongly 

Disagree 


1 

Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


39.  When  I am  using  the  DEGEM  computer,  I find  it  doesn't  work  properly. 


0 

Strongly 

Disagree 


Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 


40.  When  I get  a question  right,  the  DEGEM  computer  lets  me  know  in  an 
interesting  way. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


4 1 .  The  DEGEM  computer  graphics  are  as  good  as  other  computers. 


0 

Strongly 

Disagree 


Disagree 


2 

Neutral 


3 

Agree 


4 

Strongly 

Agree 


5 

Do  Not 
Know 
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42,  The  fact  that  the  DEGEM  computer  terminal  screens  use  green  only  rather 
than  several  colors  disappoints  me. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


43.  The  way  the  DEGEM  computer  asks  questions  about  school  subjects  is 
different  than  the  way  the  textbooks  ask  questions. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 

Disagree  Agree  Know 

44.  The  way  the  DEGEM  computer  asks  questions  confuses  me. 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 

Disagree  Agree  Know 


The  best  thing  about  the  DEGEM  computer  is 


The  worst  thing  about  the  DEGEM  computer  is 


COMMENTS: 
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THE  DEGEM  SYSTEM 
Teacher  Perspectives 


MacRosch  Educational  Consulting  Services  Inc.  has  been  contracted  by  Alberta 
Education  to  evaluate  several  aspects  of  the  DEGEM  CAI  computer  system  in  your 
school.  Consequently,  we  are  interested  in  the  teachers'  perceptions  as  to  how  well 
the  computer  system  functions.  The  results  of  this  survey  will  be  compiled  and 
analyzed  as  part  of  the  evaluation.  Please  note  that  none  of  the  information  obtained 
will  be  released  in  other  than  report  form,  and  that  no  individual  who  completes  the 
survey  will  be  identified. 

On  the  following  pages  are  a series  of  statements  regarding  the  various  aspects  of 
the  DEGEM  system  in  your  school.  For  your  convenience,  at  the  top  of  each  page 
is  a rating  scale  with  the  numbers  from  0 to  5 with  their  meanings.  The  meaning  of 
each  number  is  as  follows: 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 

Response  5,  "Do  Not  Know",  should  also  be  selected  if  the  question  is  not 
applicable  to  your  situation.  After  you  have  read  the  statement  and  decided  which 
number  most  accurately  describes  what  you  think  about  the  statement,  fill  in  the 
appropriate  circle  on  the  answer  sheet  provided.  Whenever  possible,  please  avoid 
the  use  of  the  "Neutral"  (2)  response.  Please  use  an  HB  pencil  when  completing 
this  survey.  There  is  no  need  to  identify  yourself;  however,  you  may  do  so  if  you 
wish.  There  are  opportunities  throughout  for  you  to  make  any  written  comments 
which  will  assist  us  in  obtaining  an  accurate  perception  of  the  DEGEM  system  as 
you  see  it.  Thank  You. 
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PART  I - Background  Information 

Find  the  box  on  the  answer  sheet  labelled  "SEX";  indicate  your  gender  by  filling  in 
the  appropriate  circle. 

Find  the  box  labelled  "GRADE  OR  EDUC";  indicate  the  grade  you  teach  by  filling 
in  the  appropriate  circle. 

Find  the  boxes  under  "IDENTIFICATION  NUMBER": 

Box  "A"  - Using  the  same  rating  scale  as  mentioned  above,  answer  the 
following:  "I  am  computer  literate."  Fill  in  the  appropriate  circle. 

Box  "B"  - Indicate  your  years  of  teaching  experience  by  filling  in  the 
appropriate  circle. 

Box  "C"  - Indicate  your  years  of  administrative  experience  by  filling  in  the 
appropriate  circle. 

This  question  is  designed  to  find  out  which  program  you  are  currently  using  with 
your  students.  Continue  in  the  box  labelled  "IDENTIFICATION  NUMBER".  If 
the  answer  is  NO,  fill  in  the  "0".  If  the  answer  is  YES,  fill  in  the  "1".  Which 
programs  do  you  use: 


Box  "E"  - Math 


Box  "F"  - English 
Box  "G"  - Technical  English 
Box  "FT  - Reading  Comprehension 
Box  "I"  - Typing 


Box  "J"  - Programming 


Ill 


Find  the  boxes  under  "SPECIAL  CODES": 

Box  "O"  - If  you  are  at  Kehewin  School,  fill  in  the  circle  "zero"  (0).  If  you 
are  at  Saddle  Lake  School,  fill  in  the  circle  "one"  (1). 

Box  "P"  - If  you  are  an  administrator,  fill  in  the  circle  "zero"  (0).  If  you  are  a 
teacher,  fill  in  the  the  circle  "one"  (1).  If  you  are  a systems  manager,  fill  in 
the  circle  "two"  (2).  If  you  are  a student,  fill  in  the  circle  "three"  (3). 


PART  II  - General  Questions 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 

1 . The  DEGEM  system  is  an  educational  asset. 

2 . The  DEGEM  system  makes  my  job  at  school  easier. 

3 I would  like  to  see  the  DEGEM  system  used  in  other  core  subjects  like  social 
studies  and  science. 

4.  I would  like  to  see  the  DEGEM  system  used  in  complementary  subject  areas 
like  art  and  music. 

5 . The  DEGEM  system  speeds  up  the  learning  process. 

6.  The  DEGEM  system  frustrates  me. 

7 . The  DEGEM  system  does  everything  I thought  it  would. 

8 . The  DEGEM  system  is  great! 

9 . We  should  have  more  DEGEM  computers  in  our  school. 
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10.  The  DEGEM  computer  works  the  way  it  is  supposed  to  all  the  time. 

11.  I think  the  DEGEM  system  wastes  time. 

PART  III  - Specific  Questions 

Smdent  Aghi^verngnl 

12.  The  DEGEM  system  has  contributed  to  my  students'  overall  basic  arithmetic 
skills. 

1 3 . The  DEGEM  system  has  contributed  to  my  students'  overall  problem-  solving 
skills. 

The  DEGEM  system  has  contributed  to  my  students'  overall: 

14.  Reading  comprehension  skills. 

15.  Vocabulary  skills. 

16.  Decoding  skills. 

1 7 . Reading  fluency. 

1 8 . Grammar  skills. 

1 9 . Punctuation  skills . 


20.  Spelling  skills. 
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The  way  that  the  DEGEM  System  has  contributed  the  most  to  the  achievement  of 
my  students  is  that  it 


One  way  to  improve  on  what  the  system  does  would  be  to 


COMMENTS. 


Diagnostics 

21.  I think  that  the  DEGEM  system  is  able  to  adequately  diagnose  the  problems 
my  students  have. 

22.  The  DEGEM  diagnosis  is  adequate  for  my  weaker  students. 

23.  The  DEGEM  diagnosis  is  not  adequate  for  my  average  students. 

24.  The  DEGEM  diagnosis  is  adequate  for  my  stronger  students. 

25 . The  DEGEM  diagnosis  supports  (is  consistent  with)  other  information  I have 
on  my  students. 
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26.  The  DEGEM  diagnosis  is  not  consistent  with  my  impressions  of  my  students. 

27.  The  DEGEM  diagnosis  helps  me  in  attending  to  the  educational  needs  of  my 
students. 

28 . I understand  how  to  use  the  DEGEM  printout  to  diagnose  my  students' 
needs. 

29.  Information  that  DEGEM  gives  me  on  my  students  is  not  compatible  with  the 
information  I get  from  the  CTBS. 

30.  The  DEGEM  system  ranks  students  in  the  same  order  I do. 


The  greatest  strength  in  the  DEGEM's  diagnostic  ability  is  that 


The  area  in  diagnosis  that  causes  me  the  most  difficulty  is 


COMMENTS 
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In service 

The  questions  in  this  section  are  divided  into  two  parts.  The  questions  in  Part  A 

refer  to  the  initial  training  you  received  from  the  DEGEM  staff.  Part  B questions 

refer  to  any  additional  training  you  may  have  received  from  ”in-house”  personnel. 

Part  A -DEGEM  Staff 

31.  I was  satisfied  with  the  level  of  training  I received  regarding  the  DEGEM 
computer  system. 

32.  I found  my  comfort  level  with  the  DEGEM  system  was  enhanced  by  the 
training  I received. 

33.  I saw  new  ways  to  deliver  the  curriculum  to  my  students  using  the  DEGEM 
system. 

34.  The  DEGEM  system  has  worked  as  well  they  told  me  it  would. 

35.  As  a result  of  the  training,  I have  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  HARDWARE. 

36.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  SOFTWARE. 

37.  The  training  has  provided  me  with  adequate  knowledge  of  the  operation  of  the 
DEGEM  record  keeping  functions. 

38.  As  a result  of  the  training,  I have  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  authoring  system. 
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39.  As  a result  of  the  training,  I have  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  student  diagnostic  system. 

40.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of  how  to  register 
my  class  on  the  DEGEM  system. 

4 1 . Since  I was  training  on  the  DEGEM  system  I have  found  new  uses  for  the 
system  that  were  not  covered  in  the  training. 

42.  There  were  major  content  areas  not  covered  in  the  training. 

43.  I suffered  from  information  overload  during  the  training. 

44.  I found  the  pacing  of  the  training  to  be  inappropriate. 

45.  I was  satisfied  with  the  expertise  of  the  individuals  who  did  the  training. 

Part  B - Lxx:al  Staff 

46.  I would  not  find  periodic  upgrading  useful. 

47 . I found  my  comfort  level  with  the  DEGEM  system  was  enhanced  by  the 
training  I received. 

48.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  HARDWARE. 

49.  As  a result  of  the  training,  I have  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  SOFTWARE. 

50.  The  training  has  provided  me  with  adequate  knowledge  of  the  operation  of  the 
DEGEM  record  keeping  functions. 
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51.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  authoring  system. 

52.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  student  diagnostic  system. 

53.  Asa  result  of  the  training,  I have  an  adequate  knowledge  of  how  to  register 
my  class  on  the  DEGEM  system. 

54.  I suffered  from  information  overload  during  the  training. 

55.  I found  the  pacing  of  the  training  to  be  inappropriate. 

56.  I was  satisfied  with  the  expertise  of  the  individuals  who  did  the  training. 

57.  I understand  the  intended  use  of  the  DEGEM  system  in  my  school. 

One  thing  I would  like  to  have  seen  done  to  improve  the  training  would  have  been 

to 


One  thing  that  could  have  been  left  out  of  the  training  was. 


COMMENTS 
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Thinking  Skills 

58.  Since  my  students  have  been  on  the  DEGEM  system,  I have  noticed  an 
improvement  in  their  thinking  skills. 

59.  Since  my  students  have  been  on  the  DEGEM  system,  I have  not  noticed  any 
changes  in  the  way  they  approach  problems  they  encounter. 

60.  Since  my  students  have  been  on  the  DEGEM  system,  I have  noticed  that  my 
stronger  students  approach  problems  they  encounter  in  a superior  way. 

6 1 . Since  my  students  have  been  on  the  DEGEM  system,  I have  noticed  that  my 
average  students  approach  problems  they  encounter  in  a superior  way. 

62.  Since  my  students  have  been  on  the  DEGEM  system,  I have  noticed  that  my 
weaker  students  approach  problems  they  encounter  in  a superior  way. 

63.  My  students  are  better  able  to  sequence  since  using  the  DEGEM  system. 

64.  My  students  who  are  better  able  to  sequence  benefit  more  from  the  DEGEM 
program. 


COMMENTS. 
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Teacher  Application 

65 . The  DEGEM  system  allows  me  to  provide  individualized  instruction  for  my 
students. 

66.  I individualize  instruction  for  more  of  my  students  since  using  the  DEGEM 
system. 

67.  In  my  classroom,  students  receive  less  individualized  instruction  now  than 
they  did  prior  to  the  arrival  of  the  DEGEM  system. 

68 . I individualize  instruction  for  each  of  my  students  in  the  regular  classroom 
based  upon  the  DEGEM  system  results. 

69.  The  DEGEM  system  does  not  assist  me  in  diagnosing  my  students'  strengths. 

70.  The  DEGEM  system  does  not  assist  me  in  diagnosing  my  students' 
weaknesses. 

7 1 . The  DEGEM  system  provides  me  with  information  that  allows  me  to 
remediate  my  students'  deficits. 

72.  The  DEGEM  system  is  able  to  remediate  my  students'  deficits. 

73.  lam  capable  of  assigning  my  student's  entry  level  tasks  on  the  DEGEM 
system. 

74.  When  I assign  the  student  entry  level  it  is  appropriate. 

7 5 . When  the  DEGEM  system  assigns  the  student  entry  level  it  is  appropriate. 
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76.  I prefer  to  assign  the  student  entry  level  myself  rather  than  let  the  computer  do 
it. 

77.  Since  the  DEGEM  system  has  arrived  I have  changed  the  way  I teach  in  my 
classroom. 

78.  The  DEGEM  system  is  not  compatible  with  my  teaching  style. 

79.  lam  using  my  students'  DEGEM  experience  to  enhance  learning  in  the 
classroom. 

80.  Since  using  the  DEGEM  system,  I have  more  flexibility  in  how  I can  teach. 

8 1 . Since  my  students  have  been  on  the  DEGEM  system,  I find  I have  less 
reteaching  to  do. 

82.  I use  the  DEGEM  system  information  to  determine  which  concepts  require 
reteaching  in  the  classroom. 

83.  Since  the  DEGEM  system  has  arrived,  I am  better  able  to  know  when  to 
reteach  a concept. 

84.  Since  the  DEGEM  system  has  arrived,  I am  better  able  to  know  how  to 
reteach  a concept. 

85 . Since  the  DEGEM  system  has  arrived,  I am  better  able  to  know  what  material 
to  reteach. 

86.  The  DEGEM  system  assists  me  in  evaluating  student  progress. 

87.  The  information  the  DEGEM  system  provides  assists  me  when  I am  assigning 
student  grades. 
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8 8 . The  DEGEM  diagnostic  student  results  should  be  used  to  assist  in 
determining  student  retention/promotion. 

89.  The  DEGEM  system  assists  me  in  recording  student  progress. 

90.  Report  card  marks  should  not  reflect  DEGEM  system  results. 

91.  I change  the  DEGEM  system  parameters  to  change  the  aspects  of  the  drill. 

92.  The  DEGEM  courseware  is  inappropriate  for  the  curriculum  at  the  grade  level 
I am  teaching. 

93 . The  DEGEM  courseware  is  appropriate  for  the  developmental  level  of  the 
students  in  the  grade  level  I am  teaching. 

94.  The  DEGEM  courseware  is  appropriate  to  the  cultural  needs  of  the  students  at 
the  grade  level  I am  teaching. 

95 . The  DEGEM  system  will  enhance  learning  for  all  students  in  the  school. 

96.  Since  using  the  DEGEM  system,  I find  that  my  relationship  with  my  stronger 
students  has  changed  for  the  better. 

97.  Since  using  the  DEGEM  system,  I find  that  my  relationship  with  my  weaker 
students  has  changed  for  the  better. 

98 . Since  using  the  DEGEM  system,  I find  that  my  relationship  with  my  average 
students  has  changed  for  the  better. 

99.  Since  the  DEGEM  system  has  been  in  place,  I interact  more  with  each  of  my 
students. 


122 


0 

Strongly 

Disagree 


Disagree 


1 


2 

Neutral 


Agree  Strongly 
Agree 


3 4 


5 

Do  Not 
Know 


100.  Since  the  DEGEM  system  has  been  in  place,  I interact  less  with  each  of  my 
students. 

10 1 . I present  the  topic  in  the  classroom  before  I start  my  students  on  the  DEGEM 
system. 

102.  Since  using  the  DEGEM  system,  I find  that  I do  not  have  to  do  as  much 
classroom  teaching  on  how  to  apply  the  concept. 

103.  In  arithmetic,  the  fact  that  some  of  the  answers  are  entered  "left  to  right"  and 
some  are  entered  "right  to  left"  confuses  my  students. 

104.  The  students  in  my  classroom  frequently  exceed  the  limits  of  the  DEGEM 
system,  that  is,  go  off  the  high  end  of  the  scale. 

105.  My  weaker  students  are  academically  too  weak  to  benefit  from  their  time  on 
the  DEGEM  computer. 

106.  My  average  students  are  academically  too  weak  to  benefit  from  their  time  on 
the  DEGEM  computer. 

107.  My  stronger  students  are  academically  too  weak  to  benefit  from  their  time  on 
the  DEGEM  computer. 

108.  percent  of  my  students  have  trouble  paying  attention  (are  off  task) 

during  the  10-minute  session. 

109.  I use  the  DEGEM  system  grade  equivalent  scores  in  determining  how  my 
students  are  achieving. 

1 10.  I use  the  change  in  grade  equivalent  scores  reported  by  the  DEGEM  system  to 
determine  student  achievement. 
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111.  The  way  the  DEGEM  system  asks  questions  regarding  subject  matter  is 
different  than  the  way  the  textbooks  ask  questions. 

112.  The  way  the  DEGEM  system  asks  questions  confuses  many  of  my  students. 


If  I could  change  one  thing  about  how  the  system  is  being  used  in  this  school,  I 
would  


The  thing  that  I do  not  think  should  be  changed  is 


The  most  useful  information  that  I get  from  the  DEGEM  system  is 


The  one  piece  of  information  that  the  DEGEM  system  gives  me  that  I could  do 
without  is 


COMMENTS. 
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Hardware/Software 

113.  There  are  topics  in  language  arts  I wish  the  DEGEM  system  dealt  with. 

114.  There  are  topics  in  mathematics  I wish  the  DEGEM  system  dealt  with. 

115.  Additional  topics  should  be  included  in  the  DEGEM  system  at  some  point. 

116.  My  students  move  more  quickly  through  the  topics  than  I expected. 

117.  When  I am  using  the  DEGEM  computer,  I find  it  does  not  work  properly. 

118.  When  I take  my  class  to  work  on  the  DEGEM  computer,  I find  the  whole 
system  (mainframe  computer)  is  not  working. 

119.  When  I take  my  class  to  work  on  the  DEGEM  computer,  I find  the  detachable 
keyboard  is  not  working. 

120.  When  I take  my  class  to  work  on  the  DEGEM  computer,  I find  the  student 

terminal  (including  the  keys)  is  not  working.  ^ 

121.  The  DEGEM  software  appears  to  be  outdated. 

122.  Being  unable  to  run  the  language  arts  program  and  the  math  program  at  the 
same  time  is  a disadvantage. 

123.  The  DEGEM  computer  provides  adequate  reinforcement  for  my  students. 

124.  The  DEGEM  system  graphics  are  as  good  as  other  computers. 


125.  The  fact  that  the  terminal  screens  use  green  only,  rather  than  several  colors, 
does  not  affect  my  students'  motivation. 
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If  I were  to  sum  up  the  impact  that  DEGEM  has  had  in  the  school,  I would  say 
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THE  DEGEM  SYSTEM 
Administrator  Perspectives 


On  the  following  pages  are  a series  of  statements  regarding  the  various  aspects  of 
the  DEGEM  system  in  your  school.  For  your  convenience,  at  the  top  of  each  page 
is  a rating  scale  with  the  numbers  from  0 to  5 with  their  meanings.  The  meaning  of 
each  number  is  as  follows: 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


Response  5,  "Do  Not  Know",  should  also  be  selected  if  the  question  is  not 
applicable  to  your  situation.  After  you  have  read  the  statement,  decide  which 
number  most  accurately  describes  what  you  think  about  the  statement,  and  then  fill 
in  the  appropriate  circle  on  the  answer  sheet  provided.  Whenever  possible,  please 
avoid  the  use  of  the  "Neutral"  (2)  response.  Please  use  an  ElB  pencil  when 
completing  this  survey.  There  are  opportunities  throughout  for  you  to  make  any 
written  comments  which  wiU  assist  us  in  obtaining  an  accurate  perception  of  the 
DEGEM  system  as  you  see  it.  Thank  You. 


PART  I - Background  Information 

Find  the  box  on  the  answer  sheet  labelled  "SEX";  indicate  your  gender  by  filling  in 
the  appropriate  circle. 

Find  the  box  labelled  "GRADE  OR  EDUC";  indicate  the  grade  you  teach  by  filling 
in  the  appropriate  circle. 


Find  the  boxes  under  "IDENTIFICATION  NUMBER": 

Box  "A"  - Using  the  same  rating  scale  as  mentioned  above,  answer  the 
following:  "I  am  computer  literate."  Fill  in  the  appropriate  circle. 
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Box  "B"  - Indicate  your  years  of  teaching  experience  by  filling  in  the 
appropriate  circle. 

Box  "C"  - Indicate  your  years  of  administrative  experience  by  filling  in  the 
appropriate  circle. 


Box  "D"  - Indicate  your  years  of  administrative  experience  in  this  school  by 
filling  in  the  appropriate  circle. 


Find  the  boxes  under  ’’SPECIAL  CODES”: 

Box  ”0”  - If  you  are  at  Kehewin  School,  fill  in  the  circle  ’’zero”  (0).  If  you 
are  at  Saddle  Lake  School,  fill  in  the  circle  ’’one”  (1). 

Box  ”P”  - If  you  are  an  administrator,  fill  in  the  circle  ’’zero”  (0).  If  you  are  a 
teacher,  fill  in  the  the  circle  ’’two”  (2).  If  you  are  a student,  fill  in  the  circle 
’’three”  (3). 


PART  II  - General  Questions 
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Know 

1 . The  DEGEM  system  is  an  educational  asset. 

2 . The  DEGEM  system  makes  my  administrative  job  at  school  easier. 

3 I would  like  to  see  the  DEGEM  system  used  in  other  core  subjects  like  social 
studies  and  science. 

4 . I would  like  to  see  the  DEGEM  system  used  in  complementary  subject  areas 
like  art  and  music. 
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Disagree  Agree  Know 

5 . The  DEGEM  system  speeds  up  the  learning  process. 

6 . The  DEGEM  system  frustrates  me. 

7 . The  DEGEM  system  does  everything  I thought  it  would. 

8 . The  DEGEM  system  is  great! 

9 . We  should  have  more  DEGEM  computers  in  our  school. 

1 0.  The  DEGEM  computer  works  the  way  it  is  supposed  to  all  the  time. 

11.  I think  the  DEGEM  system  wastes  time. 

PART  III  - Specific  Questions 


Student  Achievement 

1 2.  Since  the  students  have  been  using  the  DEGEM  system,  I have  noticed  an 
overall  improvement  in  basic  arithmetic  skills. 


1 3 .  Since  the  students  have  been  using  the  DEGEM  system,  I have  noticed  an 
overall  improvement  in  language  arts  skills. 


The  greatest  change  in  student  achievement  that  I have  noticed  since  the  arrival  of 
the  DEGEM  system  is 
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One  thing  about  student  achievement  that  has  not  changed  is 


COMMENTS: 


Diagnosis 

14.  The  DEGEM  diagnosis  supports  (is  consistent  with)  other  information  I have 
on  the  students. 

1 5 . The  DEGEM  diagnosis  helps  teachers  in  attending  to  the  educational  needs  of 
their  students. 

16.  Information  that  DEGEM  gives  me  on  the  students  is  not  compatible  with  the 
information  I get  from  the  CTBS. 

The  most  favorable  thing  the  DEGEM  provides  regarding  diagnosis  of  student 

achievement  is 
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Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


The  greatest  weakness  regarding  diagnosing  student  achievement  is 


Inservice 

The  questions  in  this  section  are  divided  into  two  parts.  The  questions  in  Part  A 
refer  to  the  initial  training  you  received  from  the  DEGEM  staff.  Part  B questions 
refer  to  any  additional  training  you  may  have  received  from  "in-house”  personnel 


Part  A - DEGEM  staff 

1 7 . Teachers  are  satisfied  with  the  level  of  training  they  received  regarding  the 
DEGEM  computer  system. 

1 8 . The  DEGEM  system  has  worked  as  well  as  they  told  me  it  would. 

19.  The  training  has  provided  me  with  an  adequate  knowledge  of  the  operation  of 
the  DEGEM  record  keeping  functions. 


One  thing  I would  retain  in  the  inservice  is 


One  thing  I would  do  differently  to  inservice  the  teachers  is 


131 


0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


COMMENTS 


Part  B - Local  Staff 

20.  Teachers  are  satisfied  with  the  level  of  training  they  received  regarding  the 
DEGEM  computer  system. 

21.  I understand  the  intended  use  of  the  DEGEM  system  in  my  school. 


Administrator  Perceptions 

22.  The  DEGEM  system  student  results  should  not  be  used  to 
assist  in  determining  student  retention/promotion. 

23.  The  DEGEM  system  assists  me  in  monitoring  student  progress. 

24.  Report  card  marks  should  reflect  DEGEM  system  results. 

25 . The  DEGEM  system  courseware  is  inappropriate  for  the  curriculum. 

26.  The  DEGEM  system  courseware  is  appropriate  to  the  cultural  needs  of  the 
students. 

27.  The  DEGEM  system  will  enhance  learning  for  all  students  in  the  school. 
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Disagree  Agree  Know 

2 8 . The  DEGEM  system  has  created  significant  timetabling  difficulties. 

29.  The  DEGEM  system  has  enhanced  teacher  record  keeping. 

30.  The  DEGEM  system  has  enhanced  teachers'  individualized  instruction 
planning. 

3 1 . The  DEGEM  system  has  caused  an  increase  in  teacher  requests  for 
supplementary  resource  materials. 

32.  Teachers  perceive  that  their  DEGEM  computer  time  allotment  is  adequate. 

33.  As  teachers'  classrooms  are  visited,  a general  change  in  teaching  styles  is 
noticeable. 

34.  The  DEGEM  system  printouts  enable  me  to  monitor  how  my  teachers  are 
meeting  their  students'  educational  needs. 

The  biggest  difference  that  the  arrival  of  the  DEGEM  system  has  made  to  the 

teachers  is 


The  biggest  difference  that  the  arrival  of  the  DEGEM  system  has  made  to  the 
students  is 
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Disagree  Agree  Know 


One  thing  that  I thought  would  change  but  did  not  is 


COMMENTS 


Hardware/Software 

35,  The  DEGEM  system  crashes  (does  not  function)  frequently. 

36.  The  DEGEM  hardware  often  requires  maintenance. 


COMMENTS 


If  I were  to  sum  up  the  impact  that  DEGEM  has  had  in  the  school  I would  say 
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THE  DEGEM  SYSTEM 
System  Manager  Perspectives 

On  the  following  pages  are  a series  of  statements  regarding  the  various  aspects  of 
the  DEGEM  system  in  your  school.  Under  each  statement  is  a rating  scale  with  the 
numbers  from  1 to  5.  The  meaning  of  each  number  is  as  follows: 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 

Response  5 "Do  Not  Know"  should  also  be  selected  if  the  question  is  not  applicable 
to  your  situation.  After  you  have  read  the  statement  and  decided  which  number 
most  accurately  describes  what  you  think  about  the  statement,  fill  in  the  appropriate 
circle  on  the  answer  sheet  provided.  Whenever  possible,  please  avoid  the  use  of 
the  "Neutral"  (2)  response.  Please  use  an  HB  pencil  when  completing  this  survey. 
There  are  opportunities  throughout  for  you  to  make  any  written  comments  which 
will  assist  us  in  obtaining  an  accurate  perception  of  the  DEGEM  system  as  you  see 
it.  Thank  You. 


PART  I - Background  Information 

Find  the  box  on  the  answer  sheet  labelled  "SEX";  indicate  your  gender  by  filling  in 
the  appropriate  circle. 

Find  the  box  labelled  "GRADE  OR  EDUC";  indicate  the  grade  you  teach  by  filling 
in  the  appropriate  circle. 


Find  the  boxes  under  "IDENTIFICATION  NUMBER": 

Box  "A"  - Using  the  same  rating  scale  as  mentioned  above,  answer  the 
following:  "I  am  computer  literate."  Fill  in  the  appropriate  circle. 
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Box  "B"  - Indicate  your  years  of  teaching  experience  by  filling  in  the 
appropriate  circle. 

Box  "C"  - Indicate  your  years  of  administrative  experience  by  filling  in  the 
appropriate  circle. 

Find  the  boxes  under  "SPECIAL  CODES": 

Box  "O"  - If  you  are  at  Kehewin  School,  fill  in  the  circle  "zero"  (0).  If  you 
are  at  Saddle  Lake  School,  fill  in  the  circle  "one"  (1). 

Box  "P"  - If  you  are  an  administrator,  fill  in  the  circle  "zero"  (0).  If  you  are  a 
teacher,  fill  in  the  the  circle  "one"  (1).  If  you  are  a systems  manager,  fill  in 
the  circle  "two"  (2).  If  you  are  a student,  fill  in  the  circle  "three"  (3). 


PART  II  - General  Questions 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 

1 . The  DEGEM  system  is  an  educational  asset. 

2 . The  DEGEM  system  makes  my  job  at  school  easier. 

3 I would  like  to  see  the  DEGEM  system  used  in  other  core  subjects  like  social 
studies  and  science. 

4 . I would  like  to  see  the  DEGEM  system  used  in  complementary  subject  areas 
like  art  and  music. 

5 . The  DEGEM  system  speeds  up  the  learning  process. 

6.  The  DEGEM  system  frustrates  me. 
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Disagree  Agree  Know 

7 . The  DEGEM  system  does  everything  I thought  it  would. 

8 . The  DEGEM  system  is  great! 

9 . We  should  have  more  DEGEM  computers  in  our  school. 

10.  The  DEGEM  computer  works  the  way  it  is  supposed  to  all  the  time. 

11.  I think  the  DEGEM  system  wastes  time. 


PART  III  - Specific  Questions 

0 1 2 3 4 5 

Strongly  Disagree  Neutral  Agree  Strongly  Do  Not 
Disagree  Agree  Know 


In service 

The  inservice  referred  to  here  is  the  training  provided  by  the  DEGEM  staff  to  you 
and  the  staff  in  the  school. 

12.  I was  aware  of  how  the  DEGEM  system  was  to  be  implemented  in  the  school. 

13.  I had  all  the  information  I required  before  the  DEGEM  computer  system 
arrived  in  the  school. 

14.  I now  have  all  the  information  I require  to  operate  the  DEGEM  system. 

15.  I received  an  adequate  training  experience  from  the  DEGEM  staff. 

1 6.  My  level  of  training  differed  significantly  from  that  of  the  teachers. 
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The  greatest  strength  of  the  inservice  I received  was. 


The  greatest  weakness  of  the  inservice  was, 


One  thing  that  should  be  included  in  all  future  teacher  inservices  is 


One  area  that  could  be  de-emphasized  in  future  inservices  is. 


COMMENTS 


Systems  Operator  Perspectives 


1 7 . The  DEGEM  system  has  caused  an  increase  in  teacher  requests  for 
supplementary  resource  materials. 

1 8 . Teachers  perceive  that  their  computer  time  allotment  is  adequate. 
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COMMENTS 


Hardware/Software 


19. 

20. 
21. 
22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 


The  program  allows  for  the  addition  of  new  topics. 

It  is  easy  to  add  new  topics  to  the  program. 

The  DEGEM  system  crashes  frequently. 

The  DEGEM  system  mainframe  crashes  frequently. 

The  DEGEM  system  student  terminals  malfunction  frequently. 

The  DEGEM  system  student  terminal  keyboards  malfunction  frequently. 
I frequently  find  errors  in  the  DEGEM  system  software. 

The  DEGEM  hardware  often  requires  maintenance. 

I am  able  to  handle  most  of  the  problems  that  the  system  has. 

I spend  a lot  of  my  time  fixing  system  problems  (hardware/software). 
Vendor  support  for  software  is  adequate. 

Vendor  support  for  hardware  is  adequate. 
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3 1 . Vendor  support  in  terms  of  trouble  shooting  is  adequate. 

32.  The  DEGEM  software  appears  to  be  outdated. 

3 3 . The  supporting  documentation  for  DEGEM  is  adequate  for  my  needs. 

34.  I often  use  the  authoring  features  of  this  program. 

35 . Being  unable  to  run  the  language  arts  program  and  the  math  program  at  the 
same  time  is  a disadvantage. 


One  thing  I would  suggest  to  improve  the  quality  of  the  software  is 


One  thing  that  is  very  good  about  the  software  is 


The  biggest  concern  I have  regarding  hardware  is 


The  best  thing  about  the  hardware  is, 


COMMENTS 


If  I were  to  sum  up  the  impact  that  DEGEM  has  had  in  the  school  I would  say 
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TEACHER  SURVEY 
INTERVIEW  FORM 
DEGEM  CAI 


Demographic  Data: 

Grade  Taught Gender Years  of  teaching  experience 

Subject(s)  for  which  you  are  using  the  DEGEM  computer: 

Mathematics Language  Arts Other 

Specify 

Interview  Questions: 

1 . Has  using  the  DEGEM  system  impacted  your  teaching  in  any  way? 
Yes No Explain 


How  do  you  use  the  DEGEM  computer  to  assist  you  in  your  teaching? 

Drill Remediation Diagnosis Reward Other 

Explain 


Comments 


2 . Complete  the  following  sentences: 

The  easiest  thing  about  using  the  DEGEM  system  is 


The  hardest  thing  about  using  the  DEGEM  system  is 


Overall,  using  a scale  from  1 to  10,  with  1 low  and  10  high,  how  easy  is 
the  system  to  use? 

123456789  10 
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3 .  Do  your  students  like  using  the  DEGEM  computer?  Yes No 

What  are  the  indicators? 


Are  their  certain  students  who  seem  to  like/dislike  it  more?  Yes No 

Explain 


4.  Using  a scale  from  1 to  10,  with  1 low  and  10  high,  how  much  have  your 

students  benefitted  from  their  experience  with  DEGEM? 
123456789  10 

Are  there  certain  students  who  seem  to  benefit  more?  Yes No 

Explain 

5.  Using  a scale  from  1 to  10,  with  1 low  and  10  high,  how  familiar  are  you 
with  computers? 

123456789  10 

Which  other  systems  have  you  used?  Apple Commodore 

IBM Tandy Other , if  "Other"  identify 

What  was  the  primary  purpose  for  which  you  used  the  other  system  . 


Using  a scale  from  1 to  10,  with  1 low  (compares  poorly)  and  10  high 
(compares  favorably),  how  does  DEGEM  compare? 
123456789  10 

6 .  Are  there  any  different  ways  you  would  like  to  see  DEGEM  being  used  in 
your  school? 


7 .  Finish  the  following  sentences: 

The  best  thing  about  the  DEGEM  computer  is 


The  worst  thing  about  the  DEGEM  computer  is 
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8 . How  would  you  rate  the  impact  of  the  DEGEM  computer  system  on  your 

students'  learning,  using  a scale  from  1 to  10,  with  1 none  and  10  very 
significant? 

123456789  10 

Comments  


9 . Do  you  have  any  other  comments  you  would  like  to  make  about  the  DEGEM 
computer 


Thank  you  very  much  for  your  cooperation. 


144 


STUDENT  SURVEY 
INTERVIEW  FORM 
DEGEM  CAI 

Demographic  Data: 

Grade Gender Age 

Subject(s)  for  which  you  are  using  the  DEGEM  computer: 

Mathematics Language  Arts Other 

Specify 

Interview  Questions: 

1 .  Do  you  like  working  on  the  DEGEM  computer?  Yes No 

Can  you  tell  me  more  about  that? 


2.  Pick  a number  that  tells  how  much  you  like  the  DEGEM  computer  - 1 is  low 
and  10  is  high. 

123456789  10 

3 . How  much  do  you  get  to  use  the  computer? 


Is  that  enough  time  for  you?  Yes No . Can  you  teU  me  more 

about  that? 


4 .  Finish  this  sentence.  The  best  thing  about  the  DEGEM  computer  is 


5 .  Finish  this  sentence.  The  worst  thing  about  the  DEGEM  computer  is 
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6.  Do  you  remember  when  we  came  into  your  classroom  and  asked  you 

questions  about  the  computer?  Was  there  an  important  question  about  the 
computer  we  should  have  asked?  Yes No . 

If  "Yes",  explain 

7 . If  you  could  make  one  wish  about  the  computer,  what  would  it  be? 


Thank  you  very  much  for  helping  me  today. 
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PARENT  SURVEY 
INTERVIEW  FORM 
DEGEM  CAI 

Demographic  Data  (Re:  Child(ren)): 

Grade(s)  of  Child(ren) 

Age(s)  of  Child(ren) 

Gender(s)  of  Child(ren) 

Interview  Questions: 

1 . Are  you  aware  that  your  child  uses  the  DEGEM  computer  at  school? 

Yes No 

2 . Does  your  child  talk  about  working  on  the  DEGEM  computer? 

Yes No . If  "Yes",  a little ; a lot 

3 . What  kinds  of  things  does  your  child  say  about  the  DEGEM  computer? 

Likes  working  on  it Finds  it  motivating Does  not  like  it 

Finds  it  boring Would  like  his  own  computer Would  like  to 

spend  more  time  on  it Does  not  say  anything Other  ^ : 

Specify 

4 . Have  you  noticed  any  differences  in  your  child's  attitude  toward  school  since 

he/she  began  using  the  computer?  Yes No . If  "Yes",  positive 

change ; negative  change ; other . Explain  as  necessary 


5 .  Have  you  noticed  any  changes  in  your  child's  school  marks  since  he/she 

began  to  use  the  DEGEM  computer?  Yes No . If  "Yes", 

explain 
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6.  How  would  you  rate  the  impact  of  the  DEGEM  computer  system  on  your 
child(ren),  using  a scale  from  1 to  10  with  1 low  and  10  high. 
123456789  10 

Comments 


7 . Given  what  your  child  has  said  about  the  DEGEM  computer  system,  would 

you  like  a chance  to  use  it?  Yes No . If  "Yes",  what  areas 

interest  you:  Vocational ; Upgrading ; Secretarial ; 

Other , if  "Other"  please  explain 

8 . Do  you  have  any  other  comments  you  would  like  to  make  about  the  DEGEM 

computer? 


Thank  you  very  much  for  your  cooperation. 


APPENDIX  B 


DEGEM  PRINTOUTS 
AND 

CAT  RESULTS 
-CASE  STUDIES- 
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Code  Explanation 

■*■  • correct  answer 
- ■ Incorrect  answer 
> no  response 
•*  ■ score  not  available 

Number  of  students  In  group  » 
Reference  group  (Ref)  Is 

the  national  sample  of  Grade 

LANGUAGE  MECHANICS 
No.  of  valid  sub-test  scores: 

Capital Izatlon 
Objective  27  (4  Items) 

I/Proper  Nouns/Adjectives 

01  Pronoun  I 

07  Personal  Name 

02  Proper .Adject Ive.Geog 

04  Proper . Ad Ject Ive . Geog 

Objective  28  (5  Items) 

Beginning  Words/Tltles 

03  Begin  Word. Sa lutat Ion 

09  Begin  Word. Quota t Ion 

05  Title. MOV  1 e 

08  Title. Song 

10  TItle.Poem 

Punctuation 
Objective  29  (4  Items) 

End  Marks 

1 1 Per lod 

12  Question  Mark 

15  Exclamation  Point 
24  Exclamation  Point 

. . . continued  on  back  of  sheet 
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Cod0  Explanation 

4 ■ corract  answer 
- ■ Incorrect  answer 
- no  response 
•*  m score  not  available 

Number  of  students  In  group  ■ 23 

Reference  group  (Ref)  Is 

the  national  sample  of  Grade  3.7 

Average  No. 
Items  Right 
for  Objctve 

Ciss  Ref 

MATH  CONCEPTS  ft  APPLICATIONS 

No.  Of  valid  sub- test  scores:  22 

Objective  49  (7  Items)  3.7  4.3 

Numeration 

01  Ordinal  Number . Fourth 
34  Rename  Thousands  ♦ Hunds  *■  Ones 

40  Place  Value, Hundreds 

06  Expanded  Notat Ion , Three-0 1g 

42  Number  Line  Sentence, Add 

43  Fraction  of  Group, One-Ha  If 
25  Round  3-D1g, Nearest  100 

Objective  50  (3  Items)  1.4  1.8 

Number  Theory 

03  Sequence, Subtract  Fives 
38  Odd, Even  Numbers 

41  Factor  of  4 and  8 

Objective  51  (7  Items)  4.0  3.9 

Number  Propert les/Sentences 
24  Commutative  Property, Mu  It 
32  Associative  Property, Add 
09  Identity  Element, Add 
29  Identity  Element, Mu  It 

16  Dlv  Equation, Solve  for  Box 

37  Symbol  for  True  Sentence, Mul t 

17  Greater  Than  and  Less  Than 

. . . continued  on  back  of  sheet 

Objective  52  (5  Items)  3-0 
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Average  No. 
Items  Right 
for  Objctve 

Ciss  Ref 

23 

5.4  6.3 

5.8  6.1 

g 

r 

Code  Explanation 

♦ ■ correct  answer 
- ■ Incorrect  answer 
■ no  response 
••  • score  not  available 

Number  of  students  In  group  « 
Reference  group  (Ref)  Is 

the  national  sample  of  Grade 

MATHEMATICS  COMPUTATION 

No.  of  valid  sub-test  scores; 

Objective  45  (9  Items) 

Addition 

01  Two-Dig  ♦ One-Dig. Hor 1z 

02  2 Two-0 1g, Hor 12 

14  2 Two-Dig, Hor 12 
13  3 Two-Dig, Hor 12 

15  3 Two-Dig 

29  2 Two-Dig  ♦ 2 One-Olg 

16  2 Three-Dig 

30  2 Four -Dig 

31  Up  to  Three-Dig, Hor 12 

32  4 Four-Olg 

Objective  46  (10  Items) 
Subtraction 

03  Basic  Fact. Two-Dig  - One-DI 

17  Two-Olg  - One-Dig 

33  Two-Olg  - One-Dig 

34  2 Two-Olg 

36  Three-Olg  - Mult  of  10 

04  Three-Dig  - Two-Dig, No  Regr 
20  Three-Olg  - Two-Dig 

35  Three-Olg  - Two-Olg 
19  100  - Mult  of  10 

18  Four-Dig  - Three-Dig, No  Reg 

. . . continued  on  back  of  sheet 

-Dig  - Throe-Dig, No  Regr 
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MATHEMATICS  COMPUTATION 

45  Addition  c 

46  Subtraction  1 

47  Multipl ication  1 

MATH  CONCEPTS  » APPLICATIONS 

49  Numeration  1 

50  Number  Theory  1 

51  Number  Proper t ies/Sentences  1 

52  Common  Scales  1 

53  Measurement/Graphs  1 

56  Geometry  1 
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EVALUATION  REPORT 
DEGEM  - TOAM  COMPUTER  SYSTEM 
Kehewin  and  Saddle  Lake 


PROGRAM  DESCRIPTION 

The  DEGEM  - TOAM  is  an  integrated  system  made  up  of  a forty  megabyte  miniframe  computer,  a 20  megabyte  tape 
backup,  24  to  26  terminals  with  detachable  keyboards  and  a printer.  The  accompanying  TOAM  courseware  is  a 
testing  and  drill  and  practice  system  intended  to  provide  pupils  with  the  opportunity  to  practice  extensively 
at  their  own  level  and  pace.  The  drill  and  practice  is  accompanied  by  reports  for  the  teacher  which  indicate 
the  strengths  and  weaknesses  of  the  students.  These  results  are  then  used  by  the  system  to  determine  the 
level  of  the  exercises  to  be  presented  to  the  student. 

< 

The  two  curriculum  areas  under  review  for  this  evaluation  are  Arithmetic  and  English.  Each  subject  area 
drills  basic  skills  and  is  divided  into  topics  which  are  organized  according  to  levels.  The  arithmetic 
program,  divided  into  fifteen  topics,  start  at  the  beginning  of  Grade  1 and  continues  to  the  end  of  Grade  7, 

i.e.  level  10  to  79.  The  arithmetic  topics  covered  are: 

1.  numerical  systems, 

2.  addition  and  subtraction  up  to  20, 

3.  vertical  addition, 

4.  vertical  subtraction, 

5.  multiplication  and  division  up  to  100, 

6.  vertical  multiplication, 

7.  long  division, 

8.  mental  arithmetic, 

9.  equations  and  laws  of  arithmetic, 

10.  measurements, 

11.  simple  fractions, 

1 2 decimals  and  percentages, 

1 3.  negative  numbers, 

14.  word  problems,  and 

15.  divisibility,  factoring  and  powers. 

The  English  program  is  divided  into  fifteen  topics  which  starts  at  the  beginning  of  Grade  2 and  continues  to 
the  end  of  Grade  6.  (The  ESL  program  is  divided  into  ten  topics,  starts  in  Grade  4 and  continues  to  the  end 
of  Grade  7.  A separate  Reading  Comprehension  module  exists  but  is  NOT  part  of  this  evaluation.)  The  topics 
covered  in  the  English  program  are: 

. 1 . letter/word  attack,  dictionary  skills,  abbreviations; 

2.  vocabulary; 

3.  pronouns,  noun  endings,  noun  compounds,  reference; 

4.  modals/auxiliaries,  sentence  patterns; 

5.  verb  forms; 

6.  prefixes  and  suffixes; 

7.  adjectives  and  adverbs; 

8.  noun  phrases,  prepositions,  phrasal  verbs; 

9.  sentence  patterns,  evaluative  reading; 

10.  verb  usage,  detailed  information; 

1 1 . coordination,  reference; 

1 2.  punctuation,  classification,  context  clues; 

1 3.  details  and  main  ideas; 

14.  sequence;  and 

1 5.  cause  and  effect,  conclusions.  Inferences. 
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The  TOAM  educational  approach  contains  four  main  components: 

• pre-testing, 

• drill  and  practice, 

• records  of  pupil  achievement  for  the  teacher,  and 

• post-testing. 

The  first  stage,  the  pre-testing  stage,  takes  approximately  twelve  lessons.  The  system  begins  at  a fairly 
low  level  and  then  raises  or  lowers  the  level  of  the  exercises  so  as  to  determine  the  level  at  which  the 
student  successfully  solves  two  thirds  of  the  problems  on  the  first  try.  After  the  starting  level  for  each 
of  the  topics  has  been  determined,  the  drill  begins. 

The  drill  and  practice  section  of  the  program  has  students  involved  in  working  on  a variety  of  types  of 
exercises  such  as  fill  In  the  blank,  multiple  choice  and  completing  the  answer.  Some  graphics  are  used  for 
illustration.  Questions  are  selected  randomly  from  a large  data  bank  of  questions  within  each  level. 
Students  are  given  three  chances  to  answer  each  question.  After  five  questions  qp  a topic  at  a level,  the 
results  are  evaluated  and  if  successful,  the  student  is  advanced  to  the  next  level  for  that  topic.  The 
competency  level  which  must  be  achieved  is  still  two-thirds  correct  on  the  first  try.  If  unsuccessful,  the 
student  is  presented  with  another  group  of  five  questions  at  the  same  level  of  difficulty  or  may  be  dropped 
down  a level.  Results  from  each  topic  are  recorded  separately. 

The  record  keeping  system  is  for  the  teacher  and  individual  student  or  class  reports  may  be  printed  when 
required.  The  reports  inform  the  teacher  as  to  the  last  lesson  completed,  the  level  attained,  the  number  of 
questions  attempted,  the  number  correct  on  the  first  try,  the  number  of  errors,  the  average  for  the  levels, 
those  that  caused  difficulties,  the  levels  which  were  used  for  the  diagnostic  pre-test  and  the  starting  level 
for  the  drill  and  practice  for  each  topic  for  each  student.  These  reports  are  to  help  the  teacher  isolate 
any  specific  difficulties  so  s/he  can  attend  to  them  immediately. 

The  post-testing  involves  a repeat  testing  at  the  end  of  the  school  year,  of  the  same  criteria  as  were  used 
in  the  pre-testing  phase,  in  order  to  compare  the  new  level  of  achievement. 

Before  the  students  can  begin  using  the  system,  the  teacher  must  assign  each  one  with  an  identification 
number.  This  unique  number  must  then  be  entered  by  the  student  for  each  new  session,  in  order  to  access 
his/her  work  assignment,  even  if  two  sessions  are  done  at  the  same  sitting.  The  student  should  retain  the 
same  identification  number  each  year  in  that  school.  ^ 

The  support  documentation  provided  by  DEGEM  Systems  Ltd.  includes:  Pedagogical  Principles  Underlying  the  TOAM 
System  (9  pages), The  DEGEM  System  General  Instruaions  for  the  User  (5  pages),  Mathematics/Arithmetic 
Teacher's  Manual  1987  (1 1 pages),  TOAM  2A  Arithmetic  Teacher's  Guide  (28  pages),  Mathematics/Arithmetic 
Exercise  Summary  1987  (91  pages),  English  Course  Teacher's  Manual  1987  (18  pages),  TOAM  2A  English  as  a 
Second/Foreign  Language  Teacher's  Guide  (1 1 pages),  and  English  Exercise  description  1987  (169  pages). 


PROGRAM  EVALUATION 

In-depth  Instructional  Design  and  Tolerance  and  Understanding  analyses  for  these  materials  would  be  very  time 
consuming  and  demand  approximately  six  weeks  on-site;  therefore  analyses  were  not  conducted.  The  following 
Instructional  Design  evaluations  are  based  on  observations  from  spending  a limited  amount  of  time  on-site  on 
the  system  and  from  comments  at  the  committee  meetings  by  the  teachers  responsible  for  the  systems  in  Saddle 
Lake  and  Kehewin.  There  is  also  no  assurance  that  the  specific  grade  levels  targeted  by  DEGEM  Systems  Ltd. 
for  concepts  in  fact  correspond  to  the  levels  specified  by  Alberta  Education  as  a Curriculum  Fit  has  not  been 
done  on  the  materials. 
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The  Language  Arts  content  is  very  sub-skill  oriented  for  drill  and  practice  applications  which  should  only 
complement  or  supplement  the  use  of  the  department  advocated  whollstlc  approach  for  teaching  Language  Arts. 
Nonetheless,  from  a remedial  point  of  view,  the  sub-skill  orientation  may  have  some  merit. 

The  systems  major  strength,  when  it  is  used  properly.  Is  its  diagnostic  tool  capability.  Another  strength 
includes  the  faa  that  the  system  can  be  expanded,  by  a highly  knowledgeable  computer  teacher,  to  accomplish 
tasks  for  which  it  is  not  currently  programmed.  For  example.  In  Saddle  Lake,  the  system  was  expanded  so  that 
the  terminals  with  keyboards  could  be  used  to  teach  keyboarding  and  rudimentary  word  processing  skills.  The 
system.  In  proper  hands,  could  become  a flexible  tool  to  help  students;  however,  likely  few  teachers  with  the 
level  of  expertise  required  would  be  available  to  oversee  one  of  these  systems. 

With  this  system  of  computers,  keyboards  are  detachable  and  therefore  optional.  The  schools  (at  least  in 
Saddle  Lake)  do  not  have  enough  keyboards  for  all  of  the  computer  terminals.  The  Arithmetic  and  English 
courseware,  prepared  at  Stanford  University  and  modified  in  Israel,  is  programmed  to  accommodate  the  fact 
that  keyboards  are  optional  hardware.  Therefore,  although  the  students  are  actively, involved  in  the  lesson, 
the  responses  generally  involve  pressing  a numbered  key  (0  to  9)  and  occasionally  one  of  the  keys  on  the 
j right  side  of  the  monitor.  This  selection  of  keys  may  be  appropriate  for  most  of  the  inputs  required  for 
! Arithmetic;  however,  in  Language  Arts,  it  is  questionable  as  to  the  kind  and  amount  of  learning  which  can  be 
I accomplished  by  pressing  a few  number  keys. 

According  to  DEGEM  Systems  Ltd.,  approximately  three  ten-minute  sessions  a week  is  enough  for  the  students  to 
benefit  from  the  courseware.  As  these  computers  are  located  in  a computer  room,  the  students  must  leave 
their  classroom  to  go  to  this  room,  log  onto  the  system,  do  their  ten-minute  session  and  return  to  the 
classroom.  The  entire  process,  while  observing  a Grade  One  and  a Grade  Four,  each  class  took  about  1 5 to  30 
minutes.  This  results  in  a significant  amount  of  time  off-task  for  the  potential  benefits  of  a ten-minute 
; session. 

j The  depth  of  praaice  per  session  varies  from  student  to  student  as  the  student  controls  the  program's  pace 

j by  providing  a response.  Some  students  may  cover  very  little  content  during  one  session  and  others  may 

I complete  many  exercises. 

j Questions  are  effectively  randomized  and  because  of  the  size  of  the  data  banks,  students  are  not  likely  to  be 

j presented  with  the  exact  question  twice  in  a session  unless  they  have  incorrectly  answered  the  question 

earlier.  Immediate  textual  feedback  with  limited  low  resolution  graphics  are  included.  Positive  feedback 
for  correct  answers  on  the  first  try  include  "Excellent"  and  "Very  Good".  Satisfactory  quantitative  feedback 
I is  provided  at  the  end  of  a session  and  indicates  to  the  student  the  number  of  questions  attempted,  the 
[1  number  answered  correctly  on  the  first  try  and  the  number  correct  on  the  second  or  third  attempt.  Although 

[j  thriee  attempts  are  allowed  per  question,  only  when  the  first  attempt  is  correct  is  the  answer  counted  as 

correa.  If  the  correct  answer  is  not  provided  by  the  third  try,  the  program  merely  supplies  it. 

Ij  Unfortunately,  if  the  student  accidentally  presses  a wrong  key,  the  input  is  automatically  considered  as  a 
jl  wrong  answer.  The  criteria  used  for  making  students  back  up  is  based  on  two  factors:  not  achieving  the  cut 

off  percentage  on  a set  and  having  a number  of  consecutively  incorrea  answers.  Although  identification 
numbers  are  assigned,  one  could  deliberately  access  someone  else's  work  by  entering  a code  similar  to  their 
own.  The  person's  name  appears  on  the  monitor  and  the  student  simply  presses  the  green  arrow  key  to  proceed. 

I This  is  especially  true  if  all  of  the  identification  numbers  are  assigned  sequentially. 

j Some  of  the  content  has  been  incorrectly  programmed,  with  parts  of  sections  and  even  entire  sections 
providing  inaccurate  answers.  This  is  detrimental  to  the  student  who  is  certain  s^e  is  providing  the 
; correct  answer,  yet  it  Is  considered  wrong  by  the  program.  As  a result,  in  one  school,  the  sections  found 

I containing  errors  have  been  locked  out  so  they  will  not  be  presented  to  the  students.  In  the  other  school, 

I the  errors  found  have  been  reprogrammed  accurately  and  questions  have  been  altered  to  make  them  more  relevant 
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to  this  user  group.  At  times,  the  system  erroneously  presents  system  text  written  in  Hebrew.  The  levels  of 
password  security  in  the  system  seem  inappropriate  as  students,  by  "playing"  around  could  possibly  access 
some  of  the  main  command  screens.  This  is  most  unfortunate  as  an  inquisitive  student  could  sabotage  the 
system. 

A very  high  level  of  technical  expertise  and  system  familiarity  as  well  as  a significant  amount  of  time  is 
required  from  the  person(s)  in  charge  of  this  system.  In  order  for  the  system  to  succeed,  it  relies  heavily 
on  adequate  in-servicing  for  the  teachers  Involved.  In-servicing  places  demands  on  staff  time  and  therefore 
a cost  faaor  Is  also  direaly  or  Indirectly  associated.  Furthermore,  there  is  no  way  in  which  one  can 
ensure  that  the  in-services  will  in  fact  take  place. 

The  teacher  documentation  included  with  the  system  is  minimal  although  the  English  Exercise  Description  and 
Mathematics/Arithmetic  Exercise  Summary  provide  the  teacher  with  satisfactory  descriptions  of  the  types  of 
questions  presented  at  the  specific  levels.  Unfortunately,  not  all  of  the  documentation  contains  page 
numbers  for  easy  reference.  , 

From  a technical  perspective,  the  initial  hardware  costs  were  very  substantial,  approximately  $315,CXX).00  for 
two  sites.  This  is  further  compounded  by  the  $12,000.00  per  site  annual  fee  required  for  the  maintenance 
contract.  Since  the  DEGEM  Systems  Ltd.  offices  are  located  In  Florida,  It  is  Impossible  to  have  servicemen 
respond  quickly  to  local  problems.  The  manuals  included  with  the  system  are  of  very  poor  quality,  lacking 
much  of  the  crucial  trouble  shooting  information  needed  and  even  some  of  the  simple  technical  operational 
information  which  could  assist  the  responsible  teachers  on-site  to  pinpoint  and  to  correct  the  problems.  As 
a result,  the  systems  are  sometimes  faced  with  extended  periods  of  "downtime"  when  the  system  is  not 
functioning.  One  of  the  "downtimes*  in  Saddle  Lake  resulted  from  a simple  burned  out  fuse;  however,  the 
problem  was  not  identified  until  a couple  weeks  later  and  much  hard  work  on  the  part  of  the  computer 
specialist.  It  is  unacceptable  that  such  poor  quality  documentation  would  accompany  such  a complex  miniframe 
system.  Other  technical  drawbacks  include  occasional  overheating  in  some  of  the  terminals.  This  can  be 
remedied  to  some  extent  by  removing  one  of  the  panels  on  the  computer,  thus  allowing  more  air  circulation. 
The  most  detrimental  problem  with  this  courseware  is  that  it  operates  on  an  antiquated  modified  PDP-11 
system. 

To  recapitulate,  this  system's  main  strength  lies  in  its  diagnostic  capabilities;  its  possibility  for 
expanded  use  by  a computer  specialist  to  provide  students  with  the  added  opportunity  to  learn  keyboarding  and 
word  processing  skills;  its  record  keeping/management  system  which  provides  the  teacher  with  diagnostic 
information;  and  somewhat  individualized  drill  and  praaice  sessions  to  meet  the  needs  of  the  students. 
However,  in  view  of  the  considerable  initial  costs  for  the  outdated  hardware,  the  unrealistic  maintenance  fee 
yet  unreliable  service  provided  by  Florida-based  personnel,  the  instructional  design  flaws  in  the  courseware 
identified  above,  the  DEGEM  - TOAM  educational  system  does  not  meet  our  standards. 

Evaluator:  Alberta  Education  Evaluation  Team 

Date:  Oaol^4^989 
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